← All research notes
Research notes / 2 min read

What a temporal graph benchmark can reveal

Why controlled tasks help us ask more precise questions about a model's ability to reason across time.

A strong result on a graph dataset is useful. Understanding which temporal skills produced that result is a different question.

Temporal graph models encounter relationships that change over time: interactions repeat, an event can have a delayed consequence, and relevant information can be distant in both the graph and its history. A single aggregate score can hide which of these patterns a model actually captures.

Isolate the skill

T-GRAB, a paper I coauthored, studies these questions with controlled synthetic tasks. The benchmark separates skills such as remembering repeated events, tracking delayed effects, and using long-range dependencies. Its evaluation of 11 temporal graph learning methods reveals gaps that can be difficult to diagnose from real-world benchmarks alone.

The practical value is diagnostic: when a model struggles, the task gives us a clearer account of what it struggled with.

From a score to a question

For someone building or evaluating a temporal learning system, this suggests a useful sequence:

  1. Identify the temporal pattern the application needs.
  2. Test that pattern in a controlled setting.
  3. Examine where performance changes as the task becomes harder.
  4. Return to realistic data to check whether the lesson transfers.

Synthetic tasks simplify the world. Success on them does not establish that a model will handle every real-world setting. Their value is that they make some failure modes easier to see and discuss.

Explore the work

The KDD 2026 paper, open-source code, and dataset provide starting points for exploring these questions.