Ontology Alignment Evaluation Initiative (OAEI)

What it is

OAEI is a coordinated international initiative seeking consensus on how schema- and ontology-matching methods should be evaluated. It runs controlled experimental evaluations to assess systems and techniques and to improve work on ontology alignment and matching.

Key concepts

Its goals include identifying system strengths and weaknesses, comparing technique performance, improving evaluation methods, and increasing communication among algorithm developers. Its main mechanisms are a yearly evaluation event and publication of tests and results for later analysis.

How you’d use it

Use its annual evaluations, competence-test library, and published results to compare matching approaches or analyze established evaluation tasks; the site also provides participation and dataset-submission guidance.

LLM angle

none stated

Pitfalls & lessons

The initiative emphasizes controlled evaluation and shared tests rather than unsupported comparisons between alignment systems.

Verdict

A long-running evaluation community and evidence source for ontology-matching systems, with events listed from 2004 through 2026.

Sources consulted

• http://oaei.ontologymatching.org/