# master_thesis
Software system monitoring network based on advanced methods
# Some preliminaries and notes #

# Wazuh Environment Setup with Vagrant, Docker, and VirtualBox
The setup is based on a VAGRANT env (https://gitlab.com/xavki/vagrant-files/-/tree/master/wazuh?ref_type=heads )

```markdown
## Prerequisites

Ensure that the following are installed on your system:
- **Vagrant** (Vagrant 2.4.3 or later)
- **VirtualBox** (7.1.6 Or any Vagrant-supported provider)
- **Docker Engine** (Version 27.3.1, build ce12230)
- **Docker Compose** (Included in the provisioning scripts)
- **Ubuntu & Main kernel** Ubuntu 22.04 & Kernel 6.8.0-51

Please , follow the tutorials for installation , this can be a tough process otherwise : 
https://www.virtualbox.org/wiki/Linux_Downloads
You check whether or not a virtual machine can be built manually.
Then install vagrant : 
https://developer.hashicorp.com/vagrant/tutorials/get-started/setup-project
Test it with a basic vagrant init.


### 1. Clone the Repository

Clone the repository containing your Vagrant and configuration files:

```bash
git clone <repository-url>
cd <repository-directory>
```

### 2. IP Address Configuration

Before launching the environment, modify the `variables.sh` file to set the correct IP addresses for each VM:

- **Wazuh Manager IP**: Used in the `ossec.conf` file to specify where the agent should report.
- **Agent IPs**: Configured in the `ossec.conf` file to establish the agent-server connection.

In `variables.sh` you will be able to modify IP adresses, names , wazuh update ect..:
```bash
# Example IP Address configuration
export MANAGER_IP="192.168.56.0"
export AGENT_IP1="192.168.56.1"
export AGENT_IP2="192.168.56.2"
```

**Note:** Ensure that these IPs are reflected in the corresponding `ossec.conf` files located in the Wazuh manager and agents:
- Wazuh Manager config: `/var/ossec/etc/ossec.conf`
- Wazuh Agent config: `/var/ossec/etc/ossec.conf` (for each agent)

Modify the `<server>` tags in `ossec.conf`:
```xml
<server>
  <address>192.168.56.0</address> <!-- Wazuh Manager IP -->
</server>
```

### 3. Vagrant Configuration

The `Vagrantfile` defines the network and system resources for each virtual machine. It will configure the Wazuh manager and two agents.

Modify the `Vagrantfile` if needed, or proceed with the default settings. 

If using the plugin vagrant-faster , CPUs and memory would be allocated accordingly to the machine specification : may be unappropriate while running vm on vscode
In our Vagrantfile, we use the following IP addresses:

    wazagent1: 192.168.56.10
    wazagent2: 192.168.56.11
    wazidx1 (Wazuh Server): 192.168.56.12

Before running vagrant up, make sure to check these IP addresses for conflicts:

ping 192.168.56.10
ping 192.168.56.11
ping 192.168.56.12

If any of these IP addresses respond, they are already in use, and you should choose different IP addresses.


## Network Configuration
By default, Vagrant creates a **NAT (Network Address Translation)** interface for internet access. This means that each VM will have at least two IP addresses:

1. **NAT IP Address**: This is typically in the `10.0.2.x` range and is used for outgoing connections to the internet. 
2. **Private Network IP Address**: This is usually in the `192.168.56.x` range and is specifically assigned for communication between your VMs and the host machine.

### Important Note
For the Wazuh server and agent configurations, you should **use the private network IP address** (the second one, e.g., `192.168.56.11`) instead of the NAT IP address. The NAT IP is not intended for inter-VM communication and might cause connectivity issues between the Wazuh server and agents.

## Example
When you run the command:
```bash
hostname -I
```
You may receive output similar to:
```
10.0.2.15 192.168.56.11
```
- **10.0.2.15**: This is the NAT IP address (not for Wazuh use).
- **192.168.56.11**: This is the private network IP address (use this for Wazuh configuration).

## Log Handling and Scenario Implementation

### Logger Script

The logger script is used to handle different levels of logging during the provisioning process. The script defines verbosity levels and a log function to handle different types of log messages.


#### VAGRANT FILE ####

## Prerequisites

Before running this Vagrantfile, ensure you have the following plugins installed to speed up Vagrant provisioning:

- `vagrant-faster`
- `vagrant-cachier`

You can install these plugins using the following commands:

```sh
vagrant plugin install vagrant-cachier
```

## Configuration Variables

The following configuration variables are defined:

- `MANAGER_CPUS`: Number of CPUs for the Wazuh manager (default: 6)
- `MANAGER_MEM`: Memory for the Wazuh manager (default: 6000 MB)
- `AGENT_CPUS`: Number of CPUs for each Wazuh agent (default: 2)
- `AGENT_MEM`: Memory for each Wazuh agent (default: 2048 MB)
- `MANAGER_IP`: IP address for the Wazuh manager (default: 192.168.56.13)
- `MANAGER_NAME`: Hostname for the Wazuh manager (default: wazidx1)
- `AGENT1_NAME`: Hostname for the first Wazuh agent (default: wazagent1)
- `AGENT2_NAME`: Hostname for the second Wazuh agent (default: wazagent2)

## Vagrant Box

The Vagrantfile uses the `ubuntu/jammy64` box.

## Nodes Configuration

The Vagrantfile defines three nodes:

1. **Wazuh Manager**
   - Hostname: `wazidx1`
   - IP: `192.168.56.13`
   - CPUs: 6
   - Memory: 6000 MB

2. **Wazuh Agent 1**
   - Hostname: `wazagent1`
   - IP: Dynamically assigned
   - CPUs: 2
   - Memory: 2048 MB

3. **Wazuh Agent 2**
   - Hostname: `wazagent2`
   - IP: Dynamically assigned
   - CPUs: 2
   - Memory: 2048 MB

## Provisioning Scripts
Each bash script used in the provisioning process includes a common header to ensure consistency and proper logging. The header includes the shebang line and comments for dynamic naming and sourcing necessary variables.

```bash
#!/bin/bash
# Typical Bash Script running in provisioning machine

# Get the script filename (without the path) for dynamic naming
if [[ -z "$1" ]]; then
  # If no argument is provided, use $0 to get the script name
  SCRIPT_NAME="${0##*/}"
else
  # If an argument is provided, use it as the script name
  SCRIPT_NAME="$1"
fi

BASENAME="${SCRIPT_NAME%.*}"  # Remove the file extension

# -------------------------------------------------
# HERE: Source necessary variables
# -------------------------------------------------

# This loads the DEBUG_LEVEL and other variables from sourcer.sh and variables.sh
source logging.sh
source variables.sh
source utils.sh
if [ "$WAZUH_TESTS" = false ] ; then
  log $DEBUG_LEVEL $VERBOSE_INFO "Wazuh tests not activated , skipping ..."
  exit 0
fi

# -------------------------------------------------
# HERE: Description
# -------------------------------------------------

log $DEBUG_LEVEL $VERBOSE_INFO "Starting script execution: $SCRIPT_NAME"
# Run all scenarios to tests & configure wazuh agents.

# -------------------------------------------------
# HERE: Define paths and other local constants
# -------------------------------------------------

### Common Provisioning

The following common provisioning scripts are run on all nodes:

- Update and install necessary packages
- Install Docker and Docker Compose
- Set up SSH keys
- Install requirements from `requirements.txt`
- Install Braintech
- Install Node Exporter
- Update `/etc/hosts`

### Wazuh Manager Provisioning

The following provisioning scripts are run on the Wazuh manager:

- Install Wazuh Server
- Push configuration files to the server
- Set up log rotation

### Wazuh Agent Provisioning

The following provisioning scripts are run on each Wazuh agent:

- Install Wazuh Agent
- Configure the agent
- Run tests
During the provisioning process, various scenarios are implemented to test and configure the Wazuh agents. The scripts iterate through directories containing scenario scripts and execute them.

- **Scenario Scripts**: Each directory in the scenarios folder contains a `.sh` script that is executed to test and configure the Wazuh agents.
- **Logging**: The logger script is used to log the execution of each scenario script, providing detailed information about the process.

This approach ensures that the provisioning process is well-documented and that any issues can be easily diagnosed through the logs.

## Triggers

### Before `vagrant up`

The script `setup_host.sh` is run before the `vagrant up` command.

### After `vagrant up`

The script `setup_host_finish.sh` is run after the `vagrant up` command.

### After `vagrant destroy`

The script `cleaner.sh` is run after the `vagrant destroy` command.

## Notes

- Ensure that the `MANAGER_IP` and `MANAGER_NAME` are identical in the Vagrantfile and `variables.sh`.
- All machines need to run to avoid issues.
- In shell provisioning, always use `chmod +x` if it's running on the VM.
- The `.sh` paths do not include `/vagrant`.

### 4. Launch the Environment

To create and launch the environment, run the following (as sudo !):

```bash
sudo vagrant up
```

This command will:
- Create virtual machines for the Wazuh manager and agents.
- Provision each machine by installing necessary dependencies (Docker, Wazuh) and configuring them via the `install_wazuh_server.sh` and `install_wazuh_agent.sh` scripts.

### 5. Access the Machines

After launching, you can SSH into any of the machines:

```bash
sudo vagrant ssh wazidx1   # Wazuh Manager
sudo vagrant ssh wazagent1 # Wazuh Agent 1
sudo vagrant ssh wazagent2 # Wazuh Agent 2
```

### 6. Stopping and Destroying the Environment

To stop all running virtual machines:

```bash
vagrant halt
```

To destroy all virtual machines and clean up:

```bash
vagrant destroy
```
# Some manual tests that you can do #
### 7. Modifying `ossec.conf` Files

After the VMs are up, SSH into each machine and edit the `ossec.conf` file if necessary.

#### Wazuh Manager (`wazidx1`)
```bash
sudo nano /var/ossec/etc/ossec.conf
```
Ensure the IP in the `<agent>` section matches the agents' IPs.

#### Wazuh Agent (`wazagent1`, `wazagent2`)
```bash
sudo nano /var/ossec/etc/ossec.conf
```
Ensure the IP in the `<server>` section matches the Wazuh Manager's IP.


### Notes

- Make sure your IP addresses are consistent across all configuration files (`ossec.conf`).
- Vagrant provisions the environment using scripts, which automate the installation and setup process.
- To get credentials for login :  see credential.txt

### Access VM on VScode

## Prerequisites

    Public/Private SSH Key Pair:
        Before proceeding, ensure that you have an SSH key pair (id_ed25519 or id_rsa) generated on your local machine. You can generate one using the command below if you don't already have it:

        bash

    ssh-keygen -t ed25519 -C "your_email@example.com"

    The private key will be stored in ~/.ssh/id_ed25519 (or id_rsa), and the public key will be in ~/.ssh/id_ed25519.pub.

## Public Key on VM:

    To access your VMs via SSH, your public key must be added to the VM. This is typically done by placing the public key (~/.ssh/id_ed25519.pub) in the ~/.ssh/authorized_keys file of the root user on the VM.
    Ensure that password authentication is disabled, and only key-based access is used for added security.

The `generate_ssh_config.sh` script is designed to **automate the process** of modifying the `~/.ssh/config` file on your local machine, simplifying SSH access to your VMs. Instead of manually entering configuration details for each machine, the script automatically generates the necessary SSH configurations for specific VMs and updates the `~/.ssh/config` file accordingly.

This means that after running the script, you'll have predefined shortcuts in your SSH configuration file for each of your VMs. These shortcuts make it easy to connect to each VM using a simple alias, without having to specify details like hostname, port, user, or SSH key manually.

Here's how the automation works:

- The script creates SSH entries for each VM (e.g., `wazidx1`, `wazagent1`, and `wazagent2`), defining essential parameters such as:
  - **Host**: The alias for the VM (e.g., `wazidx1`).
  - **HostName**: The VM's IP address (e.g., `127.0.0.1`).
  - **Port**: The port to connect via SSH (e.g., `2222`).
  - **User**: The user account for SSH access (e.g., `root`).
  - **IdentityFile**: The path to your private SSH key (e.g., `~/.ssh/id_ed25519`).
  
- The script automatically modifies the `~/.ssh/config` file to include these details, replacing existing configurations for these hosts if necessary.

This process saves time and reduces the chances of error when setting up SSH access to multiple VMs. After running the script, you can connect to any configured VM using a simple command like:
```bash
ssh wazidx1
```

Additionally, the script ensures that SSH access is set up securely by enforcing key-based authentication, disabling password authentication, and configuring SSH settings for automation.

By using this script, you can streamline the configuration process, making it easy to manage multiple VMs without manually editing the SSH config file.

Basically the scheme is the following : 
    - home host key created and stored at root ssh folder of the VM's for ssh connection from host user to VM
    - Config file generated in home ssh folder
    -> This two lines are for ssh connection using vs code
    - VM root keys created and stored at root ssh folder of the host for ssh connection from VM to host.
    - host root key created and stored at roo ssh folder of VM for final tests ?
    -> for sftp server
    
## Using SSH in VSCode

You can also access these VMs via Visual Studio Code using the Remote - SSH extension, which allows you to open a remote machine or VM in VSCode over SSH.

    Install the Remote - SSH extension:
        Go to the Extensions view in VSCode (Ctrl+Shift+X) and search for "Remote - SSH." Install it.

    Connect to a VM:
        Open the Command Palette (Ctrl+Shift+P), type "Remote-SSH: Connect to Host...", and select it.
        Type the alias of your VM (e.g., wazidx1) from the SSH config file, and you'll be connected to the VM.
        Once connected, you'll be able to use VSCode just as you would on your local machine, with access to the filesystem, terminal, and more.

    Root Access:
        Since the User in your ~/.ssh/config is set to root, you'll be connected as the root user, giving you full administrative access to the VM.

This setup allows you to use VSCode for development directly on your remote VMs without needing to open separate terminal windows, making it convenient to manage multiple machines.


## SRC FOLDER 
# Wazuh Malware Detection and Response System

## Project Overview

This project is a modular system for managing remote agents with specialized functionalities. Each agent type is organized hierarchically, inheriting behaviors and attributes from common base classes. The structure ensures modularity, readability, and extensibility, with each module dedicated to specific tasks or types of agents.

---

## Project Structure
To get the project structure , you can use the function : 
    save_hierarchy_to_file 
In utils.
### **Hierarchy**
- **Typo** > **Command** > **Agents**   **Dependencies (of Agents)** > **Agent** > **Remote** > **VMAgents** > ...
  - **Utils**: Contains reusable functions for remote operations.
  - **Variables**: Contains global variables for the project.
  - **Dependencies**: Daughter modules of the `Remote` class, providing additional functionalities.
  - **Agents**: Compiles all common classes related to agents.

### **Folder Structure**

```plaintext
src/
├── agents/
│   ├── agent_types/
│   │   ├── attacker/
│   │   │   └── attacker.py
│   │   ├── defender/
│   │   │   └── defender.py
│   │   ├── evaluator/
│   │   │   └── evaluator.py
│   │   └── manager/
│   │       └── manager.py
│   └── vmagents/
│       ├── fim.py
│       ├── log.py
│       └── ossec_conf.py
├── dependancies/
│   ├── llm_integration.py
│   ├── graph.py
│   └── command.py
├── utils.py
├── variables.py
└── main.py

Class Tree

The following class tree diagram illustrates the hierarchical structure of the project:

Typography: defining logs of the software and rule of coding.
    │
Command: to execute bash command from python.
    │
Agents: Mother class of any kind of agents (VM or no ie: Evaluator, Attacker etc).
    │
Evaluator: LLM monitor.
    │
Remote: Remote functions to monitor VM.
    │
Log: Handle logs from Wazuh.
    │
Xml_handler: Generate useful xml files (ie rules/decoder).
    │
Ossec.conf: Generate ossec.conf.
    │
Fim: File integrity monitoring.
    │
Yara.
    │
Vmagent.
    │
Manager.
    │
Defender.
    │
Attacker.

Valid Agent Types

Agent types must follow the folder structure under /agents/agent_types/. Each type must be imported using the following convention:

from agents.agent_types.agent_type.agent_type import AgentType

Core Components
Classes

    Agents
        Compiles common functionality for all agents.
        Represents a collection of Agent instances.

    Agent
        Base class for all agents.
        Inherited by specific agent types (e.g., Attacker, Defender, Evaluator).

    Remote
        Parent class for all VM-based agents.
        Handles remote functionality such as file management and command execution.

    VMAgents
        Subclasses of Remote, specific to VM-based agents.
        Located in /agents/vmagents/.

    Dependencies
        Provides additional functionality required by Remote.
        Includes command execution, LLM integration, and graph-based operations.

Modules

    Utils: Contains reusable functions to facilitate remote operations.
    Variables: Stores global constants and configurations.

Guidelines for Code Structure

    Class Organization
        Each class resides in its own file.
        Files are named in lowercase (classname.py).
        Class names use PascalCase (ClassName).

    Folder Structure
        Daughter folders must contain subclasses of the parent folder's classes.
        Example:
            /agents/ contains Agents class.
            /agents/agent_types/ contains subclasses of Agent.

    Function Documentation
        Each function must be detailed with:
            Argument names and types.
            Return types.
            Brief description of functionality.

    Agent Types
        New agent types must adhere to the folder structure:
            /agents/agent_types/<agent_type>/<agent_type>.py
        Agent classes inherit from Agent or Remote (if VM-based).
# LOGS
In typography , logs are defined in the following way : 
    For each agent a log record with full error .
    A general log with only the ID of the error.
