CICAPT-IIOT: A Provenance-Based APT Attack Dataset for IIOT Environment by Erfan Ghiasvand, v. 2

CIC AP IoT: A Provenance-Based APT Attack Dataset for IoT Environment - Summary

Key Concepts:

• IoT/IIoT (Industrial Internet of Things): Interconnected network of sensors, instruments, and devices with industrial applications.
• APT (Advanced Persistent Threat): Stealthy, long-term cyber attacks aimed at gaining unauthorized access to a victim's network or computer.
• Provenance Data: Metadata describing the production process of a system resource or process, represented as a directed acyclic graph (DAG).
• CIC AP IoT: A provenance-based APT attack dataset for IoT environments, combining network data and provenance logs.
• Micro Attack Framework: A knowledge base of adversary tactics and techniques based on real-world observations.
• Caldera: An automated adversary emulation system.
• Node2Vec: A network embedding technique used to generate vector representations of nodes in a graph.
• Dwell Time: The average time an APT campaign goes undetected.

1. Introduction: Why Data Matters

• The need for effective Intrusion Detection Systems (IDS) to face cyber threats requires good datasets.
• Existing datasets often fail to reflect the complexity of modern APTs, especially in industrial IoT.
• CIC AP IoT aims to provide a realistic dataset in an IoT setting, combining simulated and physical devices, networks, and scenarios.
• The dataset comprises network data and provenance logs to facilitate the development of machine learning solutions.
• Challenges in creating the dataset include capturing data, simulating real-world conditions, and ensuring data quality.

2. Background: IoT, APTs, and Provenance Data

• IoT/IIoT: An interconnected network of sensors, instruments, and other devices networked together for industrial applications (manufacturing, energy management, transportation, logistics).
  • Benefits: Improved efficiency, automation, data-driven decisions.
  • Challenges: Data privacy, scalability, and security concerns.
• Security Concerns in IIoT: Attractive targets for cyber attacks due to integration into critical infrastructure (power plants, nuclear plants).
  • Examples: Stuxnet, attacks on Ukrainian power grids.
• APT (Advanced Persistent Threat): A stealthy and persistent type of cyber attack.
  • Characteristics:
    • Remain undetected for extended periods.
    • Gain unauthorized access.
    • Full spectrum of intelligence gathering and attacking techniques.
    • Often state-sponsored.
    • Specific objectives.
    • Low and slow approach.
  • Dwell time: 70 to 200+ days (average time an APT goes undetected).
  • Phases: Initial compromise, establishing a foothold, privilege escalation, internal reconnaissance, lateral movement, maintaining presence, completing mission (data exfiltration, disruption).
• Provenance Data: Information or metadata describing the production process of an end product (system resource, process).
  • Representation: Directed Acyclic Graphs (DAGs) using the W3C PRO data model.
  • Core concepts: Types/nodes and relations/edges.
  • Nodes are connected based on causality relationships, making it effective for detecting APTs.
  • Example: A three-hour experiment provenance graph with benign (blue) and malicious (red) nodes.

3. Motivations and Research Questions

• Lack of comprehensive datasets for studying APTs in industrial IoT.
• Escalating threat of APTs as IIoT is integrated into more industrial applications.
• Research questions:
  • How to develop a comprehensive APT dataset including essential information for facing attacks in IT settings?
  • How to use provenance data to improve the detection of APTs?
  • What methods can be used to utilize provenance data for detecting and improving the detection of APTs?

4. CIC AP IoT Data Set Generation Process

• Test Bed Overview:
  • Virtual components: Host machine, two victim VMs (VM1, VM2), two attacker VMs.
    • VM1 (Edge Gateway): Calra client, local scattera, management device, MQTT subscriber, connected to the internet.
    • VM2: MQTT publisher, host of scatter VR.
  • Physical components: Two Raspberry Pis (PLC, wireless access point), camera, flat sensor, mobile phones for on-site supervision.
  • Network simulator: NS3 used to connect all components (physical and virtual).
• Attack Emulation Plan:
  • Based on AP29 emulation plan from the Micro Attack Framework.
  • Utilized Micro Attack Framework TTPs and Micro Caldera on VM1.
  • Emulation plan included: Initial access, reconnaissance, gaining access, lateral movement, persistence, and data exfiltration.
• Data Collection and Labeling:
  • Collected network data (packets with real-time timestamps) and host system logs.
  • Host logs fed into Spade to generate provenance data/graphs.
  • Caldera reports provide attack metadata and process IDs of malicious processes.
  • Used Caldera reports and process IDs to label both network data and provenance data.
  • Result: A combination of cleaned and labeled provenance data and network data.

5. Data Set Overview and Features

• Experiment phases:
  • Phase 1: Four days of benign activities.
  • Phase 2: Three days of attacks.
• Data volume:
  • Benign phase: ~46,000 provenance nodes.
  • Attack phase: ~53,000 provenance nodes, >300 attack nodes.
• Extracted features from provenance data:
  • ID (node identifier).
  • Type (node/edge type).
  • From/To (source/destination node IDs for edges).
  • Path (file/directory path).
• Provenance graph visualization: Different node and edge types represented with different colors.

6. Data Set Comparison

• CIC AP IoT is generated in an IoT/IIoT setting.
• Contains both network logs and provenance/host logs.
• Covers different phases of an APT: Initial access, foothold, collection, exfiltration, C&C, persistence, discovery, credential access, lateral movement, defensive evasion.

7. Example Usage: Provenance Graph Analysis

• Focus on using the provenance graph part of the dataset.
• Data pre-processing: Separate nodes and edges, encode features (e.g., file paths).
• Graph representation: Different node and edge types.
• Node Embedding:
  • Baseline method: Node2Vec (based on Word2Vec).
  • Generates random walks on the graph.
  • Provides embeddings for each node.
• Evaluation:
  • Machine learning models with 10-fold cross-validation and 20/30 train/test split.
  • Evaluation methods:
    • Binary classification (attack/benign).
    • Multiclass classification (attack subcategory).
    • Correlated attack stage classification.
  • Evaluation metrics: Recall and F1-score.
• Results:
  • Node2Vec embeddings used for binary and multiclass classification.
  • Correlated attack stage classification results for each attack stage.

8. Q&A Highlights

• Simulating the "low and slow" approach: Attacks were randomly spread over a three-day period, with time gaps between them.
• Why recall and F1-score: The dataset is highly imbalanced, making accuracy misleading. Recall focuses on detecting attack nodes.
• Using a network simulator (NS3): Provides full control and monitoring capabilities over the network traffic.
• Insights for industrial application cybersecurity: Addresses the lack of comprehensive datasets for APTs in industrial IoT and demonstrates the value of provenance data.
• Challenges in data collection: Ensuring data realism, synchronizing data collection, and labeling provenance graphs.

9. Synthesis/Conclusion

The CIC AP IoT dataset addresses a critical gap in cybersecurity research by providing a comprehensive, realistic, and provenance-based dataset for studying APTs in industrial IoT environments. By combining network data and provenance logs, the dataset enables the development of more effective intrusion detection systems and facilitates a deeper understanding of APT behavior. The use of a network simulator and a well-defined attack emulation plan ensures data realism, while the focus on recall and F1-score as evaluation metrics addresses the challenges posed by imbalanced datasets. The dataset and associated research contribute to improving the security of critical infrastructure and mitigating the impact of APTs in the industrial sector.