ICCMA 2023 Competition Tracks
ICCMA 2023 consists of four tracks: the main track and the special approximate, dynamic, and ABA tracks. Each track is composed of multiple sub-tracks, defined by a combination of a reasoning problem and an argumentation semantics. Argumentation systems can be submitted for evaluation into any choice of sub-tracks, i.e., there is no requirement to support e.g. all semantics for a specific reasoning problem, or all reasoning problems for a specific semantics.
Main Track
Note: The focus of the Main Track is to evaluate sequential core argumentation reasoning engines available in open source. Systems combining different core reasoning engines e.g. via portfolio-style techniques, systems employing parallel computations via the use of multiple processor cores, as well as systems which will not be made available in open source are invited to the special No-Limits Track which consists of the same subtracks as the Main Track. The organizers reserve the right to move Main Track submissions to No-Limits based on case-by-case analysis.
Problem Setting
We recall the definition of Dung's Abstract Argumentation Frameworks (AFs) [Dung 95] and its semantics.
An AF is a directed graph F = (A,R) where A is a set of arguments, and R ⊆ A × A is the attack relation. For a,b ∈ A, we say that a attacks b when (a,b) ∈ R. If in addition b attacks c ∈ A, then a defends c against b. The same concepts are extended to sets of arguments: S ⊆ A attacks (respectively defends) an argument b ∈ A if there is some a ∈ S that attacks (respectively defends) b.
The relevant semantics of AFs are defined as follows. Let the range of S ⊆ A be S {a ∈ A | S attacks a}. A set S ⊆ A is
- conflict-free (S ∈ CF(F)) if S does not attack any of its elements;
- admissible (S ∈ AD(F)) if S ∈ CF(F) and S defends all its elements against all their attackers.
- a complete extension (S ∈ CO(F)) if S ∈ AD(F) and S contains all the arguments that it defends;
- a preferred extension (S ∈ PR(F)) if S is a maximal (w.r.t. set inclusion) admissible extension;
- a stable extension (S ∈ ST(F)) if S ∈ CF(F) and S attacks each argument not in S;
- a semi-stable extension (S ∈ SST(F)) if S is complete and has a minimal range (w.r.t. set inclusion) among the complete extensions;
- a stage extension (S ∈ STG(F)) if S is conflict-free and has a minimal range (w.r.t. set inclusion) among the conflict-free extensions;
- an ideal extension (S ∈ ID(F)) if S is the maximal (w.r.t. set inclusion) admissible extension that is a subset of each preferred extension.
Subtracks
The semantics included in the Main Track are complete (CO), preferred (PR), stable (ST), semi-stable (SST), stage (STG), ideal (ID).
The reasoning problems included in the Main Track are
- DC-σ: Given F=(A,R) and a ∈ A, decide whether a is credulously accepted under σ and if it is, report a σ-extension that contains a.
- DS-σ: Given F=(A,R) and a ∈ A, decide whether a is skeptically accepted under σ and if it is not, report a σ-extension that does not contain a.
- SE-σ: Given F=(A,R), return a σ-extension of F.
Ranking
Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The time limit is 1200 seconds CPU time per instance, and PAR-2 scoring is used. That is, the score for a given solver on an instance is 2 * 1200 if the solver timed out on this instance, and otherwise the CPU running time of the solver on this instance in seconds. The score for a solver on a subtrack is the sum of the solver’s scores over all instances of the subtrack. The winner is the solver with the lowest score.
No-Limits Track
The No-Limits Track is a more permissible version of the Main Track and has the same problems as subtracks. In particular, solvers are allowed to run on multiple cores and can be portfolio-based (i.e. combine the usage of several solvers). The execution environment has 8 cores and 16 threads available. The ranking is otherwise the same as for the Main Track, but wall-clock time is used instead of CPU time.
Approximate Track
Problem Setting
Abstract argumentation, as defined above. Correctness requirements and ranking are different than other tracks: incorrect solutions are simply discarded and only the number of correct solutions is taken into account. The time limit for the track is lower than for the other tracks, namely 60 seconds CPU time per instance.
Subtracks
Semantics: CO, PR, ST, SST, STG, ID. Reasoning modes: DC-σ and DS-σ. The following combinations of the semantics and reasoning modes is a subtrack: DC-{CO|ST|SST|STG|ID}, DS-{PR|ST|SST|STG}.
Ranking
Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The solver with the largest amount of correctly solved instances overa time limit of 60 seconds wins. If needed, cumulative CPU running time over solved instances is used as a tie-breaker.
Dynamic Track
Problem Setting
Abstract argumentation, as defined above. Dynamic changes to an initial AF and acceptance queries are issued by different applications via IPAFAIR, an API for incremental reasoning in abstract argumentation. Please see ipafair.py in the repository for more details.
Subtracks
The subtracks in the Dynamic Track are DC-CO, DS-PR, DC-ST, and DS-ST. Each subtrack consists of applications which call an AF solver which implements the interface, applying changes to the underlying AF between solve calls. The name of the subtrack determines the allowed reasoning task and semantics. For example, in the DS-PR subtrack, the AF solver is initialized using the preferred semantics, and only solve_skept() calls are allowed with a single query argument. Both the query argument and the underlying AF may change between these calls.
Ranking
Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The time limit is 1200 seconds CPU time per instance, and PAR-2 scoring is used. That is, the score for a given solver on an instance is 2 * 1200 if the solver timed out on this instance, and otherwise the CPU running time of the solver on this instance in seconds. The score for a solver on a subtrack is the sum of the solver’s scores over all instances of the subtrack. The winner is the solver with the lowest score.
ABA Track
Problem Setting
Assumption-based Argumentation (ABA) [Bondarenko et al 97] and the corresponding semantics are defined as follows.
An ABA framework is a tuple F = (L,R,A,‾) where
- L is a set of literals
- R is a set of rules
- A ⊆ L is a (non-empty) set of assumptions
- ‾ is a mapping from assumptions to literals, that represents a notion of contrariness
A sentence a ∈ L is derivable from a set X ⊆ A via rules R, denoted by X ⊢ a, if a ∈ X or there is a sequence of rules (r1,...,rn) such that head(rn) = a and for each rule ri we have ri ∈ R and each sentence in the body of ri is derived from rules earlier in the sequence or in X. A set of assumptions A1 attacks a set of assumptions A2 if the contrary of some a ∈ A2 is derivable from A1. A set of assumptions A1 defends an assumption a if A1 attacks every set of assumption that attacks a.
The semantics for an ABA framework can be defined as follows. Given a set of assumptions X ⊆ A,
- X ∈ CF(F) if it does not attack any of its elements;
- X ∈ AD(F) if X ∈ CF(F) and X defends all its elements;
- X ∈ CO(F) if X ∈ AD(F) and X contains every assumption it defends;
- X ∈ PR(F) if it is a ⊆-maximal admissible set of F;
- X ∈ ST(F) if X ∈ CF(F) and X attacks every assumption not in X.
Subtracks
Semantics: CO, PR, ST.
The reasoning tasks for the ABA Track are
- DC-σ: Given ABF F=(L,R,A,‾) and s ∈ L, decide whether s is credulously accepted under σ.
- DS-σ: Given ABF F=(L,R,A,‾) and s ∈ L, decide whether s is skeptically accepted under σ.
- SE-σ: Given ABF F=(L,R,A,‾), return a σ-extension of F.
The following combinations of the semantics and reasoning modes is a subtrack: DC-{CO|ST}, DS-{PR|ST}, SE-{PR|ST}.
Ranking
Each subtrack (i.e. semantics and reasoning mode combination) is ranked separately. The time limit is 1200 seconds CPU time per instance, and PAR-2 scoring is used. That is, the score for a given solver on an instance is 2 * 1200 if the solver timed out on this instance, and otherwise the CPU running time of the solver on this instance in seconds. The score for a solver on a subtrack is the sum of the solver’s scores over all instances of the subtrack. The winner is the solver with the lowest score.
References
[Dung 95] P. M. Dung, On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games. Artif. Intell. 77(2): 321-358 (1995)
[Bondarenko et al 97] A. Bondarenko, P. M. Dung, R. Kowalski, F. Toni, An Abstract, Argumentation-Theoretic Approach to Default Reasoning. Artif. Intell. 93: 63-101 (1997)