> For the complete documentation index, see [llms.txt](https://walterdrake.gitbook.io/mysite/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://walterdrake.gitbook.io/mysite/project/implemented-snort-misp-cowrie-and-machine-learning-to-bolster-network-security.md). # Implemented Snort, MISP, Cowrie, and Machine Learning to bolster network security ## Introduction Our team project focuses on deploying an IDS/IPS system to prevent simulated attack scenarios, acquiring new Indicators of Compromise (IOCs) related to malware, utilizing honeypots to detect attackers' intentions, and leveraging machine learning to identify botnets. ## Objective Our primary objective is to implement this system in real-world scenarios effectively. The key components of our approach are: * **Snort:** Used to detect and prevent Distributed Denial of Service (DDoS) attacks. * **MISP:** Facilitates the acquisition and sharing of Indicators of Compromise (IOCs). * **Cowrie:** Deployed as a honeypot to monitor and analyze attacker behavior. * **Machine Learning:** Utilized to identify and detect botnets with greater accuracy. ## Implement ### Network Architect

### Configure VMWare Our setup includes two VMware instances: one serves as the attacker, running Kali Linux, and the other is an Ubuntu Server hosting Snort, MISP, and Cowrie. Due to resource limitations, we simulate DoS attacks rather than full-scale DDoS attacks. Both systems are configured within the same local area network (LAN) using the IP range `10.81.1.0/24`. ### Stack **Snort** Snort version 3.6.0.0 is used for intrusion detection and prevention. It monitors network traffic for potential threats, such as DoS attacks, and provides real-time alerts for suspicious activities. **MISP** MISP is deployed using Docker, offering a platform for collecting, storing, and sharing Indicators of Compromise (IOCs). It helps streamline the process of threat intelligence management and enhances collaboration in the security community. #### **Cowrie** Cowrie is set up as a honeypot to simulate a vulnerable system and observe attacker behavior. By capturing and analyzing attack patterns, it provides insights into attacker tactics, techniques, and procedures. #### **Machine Learning** A machine learning model is employed to detect botnet activity by analyzing network traffic. The model improves accuracy in identifying anomalous behaviors associated with botnets, offering an additional layer of security detection. {% hint style="info" %} For the installation and setup of all components, detailed instructions are available in the resource section. {% endhint %} ## Deploy the scenarios ### Summary #### Snort We run Snort in inline mode using the command: ``` sudo snort -c snort.lua -v -Q --daq afpacket -i ens34:ens33 -A alert_fast -s 65535 -k none ``` {% hint style="info" %} The command runs Snort with a custom configuration (`snort.lua`) in verbose mode, using the afpacket DAQ method to capture packets on specified interfaces (`ens34:ens33`). It generates fast alerts (`alert_fast`), captures full packet sizes (`-s 65535`), and disables checksum validation (`-k none`) {% endhint %} In the `snort.lua` configuration file, we define various variables and include specific rules to apply to the system. More details on these settings will be provided when discussing them further.

#### MISP We use Docker to deploy and run the MISP instance, ensuring a streamlined and efficient setup process.

In this setup, we utilize only three feeds as examples to demonstrate MISP's functionality.

To perform specific actions, we use MISP OpenAPI to interact with the MISP instance and write Python scripts for automation. These scripts are available in my GitHub repository, which is linked in the resource section. #### Cowrie We followed a tutorial to set up Cowrie and adjusted the script to align with our specific objectives. This configuration allows Cowrie to not only detect and log intentional attack attempts but also block them before the attackers can execute their actions. #### Machine Learning We followed the methodology outlined in the paper "[Botnet Detection](https://github.com/harvardnlp/botnet-detection/tree/master)" to implement machine learning techniques for detecting botnet activity. This approach involves analyzing network traffic patterns to identify potential botnet behavior. ### Simulating DoS Attacks #### Rule We have created two rules for detecting ICMP flood and SYN flood attacks, which are included in the `ddos.rules` file. * ICMP flood ``` alert icmp any any -> $HOME_NET any (msg:"ICMP flood"; sid:1000001; rev:1; gid:1; classtype:icmp-event; detection_filter: track by_dst, count 500, seconds 5;) ``` {% hint style="info" %} This rule detects ICMP flood attacks by monitoring traffic and triggering an alert if 500 ICMP packets are received within 5 seconds to a single destination. {% endhint %} * SYN flood ``` alert tcp any any -> $HOME_NET $HTTP_PORTS (flags: *SR; msg:"Possible DoS Attack Type : SYN flood"; flow:stateless; sid:1000002; detection_filter: track by_dst, count 20, seconds 5; rev:1; gid:1;)// Some code ``` {% hint style="info" %} This rule identifies potential SYN flood attacks by looking for 20 TCP packets with SYN and/or RST flags set, targeting HTTP ports within 5 seconds to a single destination. {% endhint %} #### Alert We use Kali Linux to launch attacks with `hping3/ping` and observe the alerts generated by Snort. * ICMP Flood

* SYN Flood We host a server using Python to simulate a DoS attack on the web server.

#### Prevent To prevent attacks, we use the `rate_filter` in the `snort.lua` file, which changes the action from generating an alert to dropping the malicious traffic.

{% hint style="warning" %} Although we followed the documentation, the `drop` action in Snort is supposed to drop malicious packets. However, in our case, it still appears that Snort only notifies the action as "drop" without actually dropping the packets. When we check using Wireshark, we can still see replies from the server. {% endhint %} ### MISP: Acquiring IOCs #### Fetch feeds To automate the updating of newly acquired IOCs from enabled feeds, we set up a cron job using the crontab tool. ``` 0 1 * * * /usr/bin/curl -XPOST --insecure --header "Authorization: APIKEY" --header "Accept: application/json" --header "Content-Type: application/json" https://localhost/feeds/fetchFromAllFeeds ``` {% hint style="info" %} The cron job runs daily at 1:00 AM and uses `curl` to send a POST request to the MISP instance's API endpoint (`/feeds/fetchFromAllFeeds`). It includes the necessary headers, such as the API key, to fetch updates from all enabled feeds securely. {% endhint %} #### Download rules To export rules for Snort, we developed a Python script that uses MISP OpenAPI to retrieve and save the rules locally. The script also preprocesses the rule syntax to ensure compatibility with Snort.

I also created a crontab to automate this process, ensuring that the Python script runs at scheduled intervals to export and preprocess the Snort rules automatically. ``` 30 2 */2 * * sudo python3 ~/script/down_rule.py ``` {% hint style="info" %} The crontab runs every two days at 2:30 AM, executing the Python script (`down_rule.py`) with `sudo` to download and preprocess the Snort rules automatically. {% endhint %} #### Export rules To manage the rules added to Snort, we created a script that compares the currently applied rules with any updated rules, ensuring that changes are properly tracked and implemented.

### Cowrie: Detecting Intentional Attacks We use Nmap to scan for open ports, mimicking the behavior of an attacker to observe how Cowrie detects and logs these scanning attempts.

After performing the port scan, we connect to the honeypot and check the logs to analyze the attacker's behavior and ensure that Cowrie is accurately logging and detecting the activities.

Upon checking the logs, we can see that a shell was opened, indicating that the attacker successfully interacted with the honeypot, allowing us to analyze their actions.

#### Export rules: To extract the IP addresses of attackers, we created a script that reads the `cowrie.json` file and identifies the IPs associated with the `cowrie.session.connect` event ID, which logs the connection attempts by the attackers.

Contribution

[azhlm](https://github.com/azhlm)

### Machine Learning Traditional rule-based methods often fail to detect coordinated attacks, such as botnets, as they cannot analyze the relationships between multiple requests. Similarly, simple machine learning models that assess individual requests may perform well on controlled datasets but struggle in real-world scenarios. Effective detection requires analyzing a host's behavior within the context of larger request snapshots, ranging from 1,000 to 10,000 requests. Graph Neural Networks (GNNs) and Graph Attention Networks (GATs) are highly effective in this context, as they capture the relational structure and dependencies within the network. This capability enables them to deliver superior performance when applied to real-world systems. #### Key Benefits * **Relational Insights:** GAT effectively captures host interactions within a request snapshot, enabling the identification of coordinated botnet behavior. * **Focused Detection:** The attention mechanism emphasizes critical connections that are indicative of potential attacks, improving detection accuracy.

A graph plot depicting a normal network during a centralized botnet attack.

A graph plot illustrating the network after the model predicts botnet activity.

Contribution

[F1xedbot](https://github.com/F1xedbot)

### Additional We integrated the OpenAppID module with Snort to improve application detection by analyzing traffic patterns. OpenAppID helps Snort identify and categorize applications, allowing for more precise detection of threats based on specific application behaviors, enhancing overall network security. ``` alert tcp any any -> any any ( msg:"Google Detected"; appids:"Google"; sid:1000006; metadata:policy security-ips alert; ) ```

{% hint style="warning" %} When applying AppID in rules, Snort should operate in passive mode, focusing solely on detecting and identifying application traffic without actively blocking it. {% endhint %} ## Resources {% hint style="success" %} **Snort** * [Snort 3 (IPS) - Installation, Configuration and creating Local Rules](https://www.youtube.com/watch?v=5eB4SnT7dtg) * [How to Install & Configure Snort on Ubuntu Linux?](https://www.zenarmor.com/docs/linux-tutorials/how-to-install-and-configure-snort-on-ubuntu-linux) * [Snort 3 Rule Writing Guide](https://docs.snort.org/welcome#snort-3-rule-writing-guide) **MISP** * [MISP - User Guide A Threat Sharing Platform](https://www.circl.lu/doc/misp/) * [MISP Automation API](https://www.misp-project.org/openapi/) **Cowrie** * [Setup a honeypot and catch hackers for FREE | cowrie tutorial](https://www.youtube.com/watch?v=jo-eaptc9Bw) * [How to Hack the Hackers | Cowrie Honeypot](https://www.youtube.com/watch?v=m7ZmwjyhzHU) * [Welcome to Cowrie’s documentation](https://docs.cowrie.org/en/latest/index.html) **Script** * [MISP-Cowrie-Script](https://github.com/WalterDrake/MISP-Cowrie-Script) {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://walterdrake.gitbook.io/mysite/project/implemented-snort-misp-cowrie-and-machine-learning-to-bolster-network-security.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.