Combining Crowd Sensing and Social Data Mining with Agent-based Simulation

Using Mobile Agents towards Augmented Virtuality

Stefan Bosse, Uwe Engel
1University of Koblenz-Landau, Fac. Computer Science, Koblenz, Germany
2University of Bremen, Dept. Social Science, Bremen, Germany
26.9.2019

Overview

Demonstration and Presentation at http://ag-0.de

Introduction

The key concept of this work is the consideration of humans as sensors.

Human-in-the-loop Simulation integrates humans in simulation.

Fusion of Real and Virtual Worlds by an unified Agent-based approach.

Interdisciplinary Concept → Merit of social and computer science.

Agent-based Methods

Agent-based methods are established for modelling and studying of complex dynamic systems and for implementing distributed intelligent systems!

Agent-based Methods Classes
  1. Agent-based Modelling (ABM): Modelling of complex dynamic systems by using the agent behaviour and interaction model ⇒ Physical agents
  2. Agent-based Computing (ABC): Distributed and parallel computing using mobile agents related to mobile software processes ⇒ Computational agents
  3. Agent-based Simulation (ABS): Simulation of agents or using agents for simulation
  4. Agent-based Computation, Modelling, and Simulation (ABX): Combining physical and computational agents

The fourth class is the novelty introduced in this work with application to social mobility and network simulation

Crowd Sensing, Surveys, and Cloning of Digital Twins

  • Crowd sensing is performed via mobile chat bot agents

    • Survey is performed by chat bot agents that can execute dynamic dialogues via a chat dialogue platform

  • Interaction of real humans with agents with simulation world in real-time

  • Survey performed by chat bot agents (computational agents) creates Digital Twins in the virtual world from survey participants

  • Parameters P of twin behaviour model derived from survey data feedback F retrieved by a user dialogue D.

\[%% Tex \begin{array}{lcl} survey(D) & : & D \rightarrow Ag_{chatbot} \rightarrow F \\ analyse(F) & : & F \rightarrow P \\ twin(P) & : & P \rightarrow Ag_{twin} \\ simulation(M,P,X,Ag_{twin}) & : & X \times M(P) \rightarrow Ag(M) \times Ag_{twin} \rightarrow Y\\ \end{array} \]

The Concept

Merging Real and Virtual Worlds

#augvirt1

Fig. 1. (Left) Traditional field studies and agent-based simulation of social systems (Right) New combined agent-based simulation and agent-based crowd sensing enabling bidirectional data exchange

Real and Virtual Agents

There are three classes of agents:

Physical Agent

A physical agent represents some kind of physical entity. The behaviour is derived from real world and consists of mobility, interaction, and decision making
→ ABM/ABS/ABMS → Virtual World

Computational Agent

A computational agent is just mobile software that is used to perform distributed data processing, e.g., Mobile Crowd Sensing
→ ABC → Real + Virtual World

Simulation Agent

A simulation agent controls a simulation or field/survey study
→ ABC → Virtual World

Merging Real and Virtual Worlds

#augvirt2

Fig. 2. The software framework using physical and computational agents

Simulation Architecture and Framework

Some Technical Details!

Components

  1. Unified Agent Processing Platform (JavaScript, JAM);
  2. Crowd Sensing Software (Mobile App and WEB Browser using JAM);
  3. Agent-based Simulation with Internet connectivity supporting three different agent types (SEJAM providing virtualisation of JAM):
    • Physical behavioural agents representing physical entities, e.g., individual artificial humans;
    • Computational agents representing mobile software, i.e., used for distributed data processing and digital communication, and implementing chat bots;
    • Simulation agents controlling the simulation and performing simulation analysis (e.g., creation of physical and computational agents, reading and writing sensor data, accessing data bases)
  4. Chat dialogues, Chat bots and mobile agents collecting user and device sensor data;
  5. (Knowledge-based Question-Answer Systems, Natural Language Processing)

Components

Closed-loop Simulation

  • Agent Processing Platform (JAM) is programmed in JavaScript for portability and flexibility

    • Can be executed by any JavaScript engine
      (nodejs/V8, cordova, spidermonkey, jerryscript)

  • The agents are programmed in JavaScript, too!

    • A computational agent is mobile (code+data)

  • SEJAM Simulation framework provides virtualisation of the JAM platform;

    • NetLogo compatible API for physical agents (ask,create,..)!

  • Mobile and non-mobile devices can execute the JAM platform

    • Smartphones, mobile computers, servers, embedded and IoT devices → Crowd Sensing,
    • Connected with the virtual simulation world via the Internet!

Components

#simulation1

Fig. 3. Principle concept of closed-loop simulation for augmented virtuality.

Experiment

To demonstrate the augmented simulation approach combining ABMS with ABC Crowd Sensing the Sakoda model [2] was chosen as a simple social interaction and behaviour model between groups of individual humans.

It poses self-organising behaviour (emergence) and structures of social groups by segregation.

Crowd Sensing creates parametrized Digital Twins in the simulation world via surveys introducing variance

The Multi-agent System

figmas

Fig. 4. The MAS consisting of interacting physical and computational agents (S: Survey Agent, Ag: Sakoda Agent, Tw: Digital Twin from Survey)

The Multi-agent System

The crowd sensing is performed via a JAM relay and by a survey agent posing different sub-class behaviour:

Survey-Master

This agent is created, e.g., in the simulation world by the simulation controller agent. The master agent searches a relay node and stays there to service survey requests. If there are pending surveys (time limited), the survey is sent to each new node linked with the relay (mobile apps, e.g.) via a worker agent.

Survey-Requester

This agent is the crowd sourcer that searches a node occupied with a survey master agent. The requester agent passes a survey script to the master agent. If it is a authorised request, the master agent performs the survey. The results are passed back to the requester agent. The requester can either go back to its source node to deliver the survey results or it can send out survey twin agents (one for each returned survey).

The Multi-agent System

Survey-Worker

The worker agent is sent out by the master agent to search mobile nodes occupied with a chat moderator agent. The worker performs the survey (executes the script). After the survey is finished, it returns to its source node to deliver the survey results back to the master agent that delivers the results to the original requester agent.

Survey-Twin

A survey twin agent carries the results of a particular survey and migrates into the simulation world to create a digital twin agent from the survey results.

Social Clustering and Segregation

The Sakoda Interaction Model

  • It is assumed that there are different individual humans related to classes C={a,b,..}.
  • The social interaction is characterised by different attitudes [2] of an individual between different and among same groups given by four parameters:
\[%% Tex S_{ab}=(s_{aa},s_{ab},s_{ba},s_{bb}), {S_{abcd}} = \left( {\begin{array}{*{20}{c}} {{s_{aa}}}&{{s_{ab}}}&{{s_{ac}}}&{{s_{ad}}} \\ {{s_{ba}}}&{{s_{bb}}}&{{s_{bc}}}&{{s_{bd}}} \\ {{s_{ca}}}&{{s_{cb}}}&{{s_{cc}}}&{{s_{cd}}} \\ {{s_{da}}}&{{s_{db}}}&{{s_{dc}}}&{{s_{dd}}} \end{array}} \right) \]

Social Clustering and Segregation

  • The world model consists of N places xi.

    • Each place can be occupied by none or one agent of the group set C={a,b}, expressed by the variable xi={0,-1,1}, or generalised xi={0,1,2,3,4,..,n} with n groups.

  • The social expectation of an individual i at place xi is given by:

\[%% Tex f_i(x_i) =\sum_{k=1}^N J_{ik} \delta_s(x_i,x_k) \]

  • Parameter Jik is a measure of the social distance (equal one for Moore neighbourhood with distance one), decreasing for longer distances.

  • Parameter δ expresses the attitude to a neighbour place, given by (for the general case of n different groups):

\[%% Tex \delta_{s}(x_i,x_k)=\left\{ {\begin{array}{*{20}{l}} {{s_{\alpha \beta }}}&{,{\text{ if }}{x_i} \ne 0{\text{ and }}{x_k} \ne 0{\text{ with }}\alpha {\text{ = }}{x_i},\beta {\text{ = }}{x_k}} \\ 0&{,{\text{ otherwise}}} \end{array}} \right. \]

Social Clustering and Segregation

  • An individual agent agi of group a or b; is able to change its position by migrating from an actual place xi to another place xq
    • if this place is not occupied (xq=0) and
    • if fi(xq) > fi (xi).

Evaluation

#simu001

Fig. 5. (a) Simulation world consisting of 200/200 artificial agents of class a/b (blue/red squares), randomly distributed (b) Simulation world after social organisation based on mobility forming strong isolated homogeneous clusters with S=(1,-1,-1,1) and r=3 interaction radius after 200 simulation steps

Social Clustering and Segregation

#simu002

Fig. 6. Simulation world at different simulation times (500/1000/1500 steps) consisting of 200/200 a/b class agents (blue/red squares) all with S=(1,-1,-1,1) and r=3 parameter settings and additionally up to 200 digital twins (triangles with colour based on individual S/r parameters)

Conclusions

  1. Agent-based Modelling with physical agents and simulation is a powerful tool to study complex dynamic systems and their interaction

  2. Combining physical agents and computational agents couples Crowd Sensing and Simulation

  3. Crowd sensing is used to perform surveys

  4. The survey data is used to create digital twins in the simulation

    • Strength of data base
    • Introduction of variance

References

  1. S. Bosse, U. Engel, Augmented Virtual Reality: Combining Crowd Sensing and Social Data Mining with Large-Scale Simulation Using Mobile Agents for Future Smart Cities, Proceedings, Volume 4, ECSA-5 5th International Electronic Conference on Sensors and Applications 15–30 November, 2018 DOI 10.3390/ecsa-5-05762

  2. P. Medina, E. Goles, R. Zarama, and S. Rica,Self-Organized Societies: On the Sakoda Model of Social Interactions, Complexity, 2017

  3. T Leppäne, J. Á. Lacasia, Y. Tobe, K. Sezaki, and J. Riekki. 2017. Mobile Crowd Sensing with mobile agents. Autonomous Agents and Multi-Agent Systems, vol. 31, no. 1, pp. 1-35

Questions and Comments

Combining Crowd Sensing and Social Data Mining with Agent-based Simulation Using Mobile Agents towards Augmented Virtuality

Stefan Bosse1, Uwe Engel2

1University of Koblenz-Landau, Fac. Computer Science, Koblenz, Germany
2University of Bremen, Dept. Social Science, Bremen, Germany


Demonstration and Presentation at http://ag-0.de