Design Challenge

---

As a set of uncoordinated but interacting parts, multi-agent systems are a fertile ground for emergent behaviors. These systems evolve over time as the agents are updated, added and removed. Furthermore, they evolve as the environments they interact with change. This happens at the speed and scale of computation. Multi agent systems like these are increasingly common in more settings. This year’s mini challenges highlight some issues in working with such systems, derived from research into how they fail. The issues that surfaced in these challenges illustrate a few failure modes, but there are many more.
There will be a consistent need in coming years to investigate system behaviors and diagnose failure modes. These needs are the motivation for this year’s VAST challenge (and likely future VAST challenge and real-world challenges). This year’s design challenge asks you to look forward toward those other scenarios.
What tools will be needed to explore the data in such environments? Key challenges stem from:

• The speed that autonomous systems can act.
• The volume of data that such systems can produce.
• Complex interactions between autonomous parts.
• Behaviors that are unexpected because they go against norms that computers do what we tell them and nothing more.

There are many challenges you could take on in this space. These include:

• Map the system: Identifying the actors, the roles they play in the system, and how they interact with each other.
• Determining expected behaviors: Eliciting, documenting and representing what the system was intended to do, and illustrating how it deviated.
  
• Build a baseline: Complex systems exhibit many behaviors concurrently, and those behaviors may appropriately shift over time.
• Probabilistic causal reasoning: Addressing issues requires considering not just what happened but why. However, the system is likely to be only partially observed.

In all the above challenges, there is ample room for both human and automated reasoning to be applied. The design challenge encourages you to come up with innovative ways to integrate these types of reasoning.

Tasks and Questions:

This is an open design challenge. Please create a submission in markdown format that describes a design for addressing the challenges.

• You may assume detection is tractable, timely and provided in a format of your convenience.

• Working software is not required, though it is welcome if you produce it. For parts that are not implemented, please describe what further algorithms would be required for the monitoring & communication.

• Cite inspirations from other fields. You may use this data, but do not feel obligated to. If you use other data sources, please only use open available datasets.

• You may include in your submission:

  • Design evolution: A description of choices made at each step of a design’s refinement.

  • Failed ideas: Things that were experimented with that did not work out. Include a discussion of (1) why you thought they would work and (2) what you found out through your evaluation process.

  • Hand-made concept sketch.

The submission format is a markdown document including images and a video. Please use your judgement on how to best communicate your ideas, however a rough length of 4 pages including images is requested.