A comprehensive security monitoring solution that combines external threat intelligence with real-time local network monitoring. The application provides a unified web dashboard for threat intelligence analysis and local security monitoring.
- Multi-source Integration: AbuseIPDB, VirusTotal, Shodan, and HttpBL
- IP Reputation Checking: Real-time IP reputation analysis
- Malware Detection: File hash analysis and malware scanning
- Network Intelligence: Port scanning and service detection
- MITRE ATT&CK Integration: Threat correlation with MITRE framework
- Real-time Connection Tracking: Monitor all network connections
- Process Monitoring: Track system processes and their network activity
- YARA-based Malware Detection: Local malware scanning using YARA rules
- Threat Correlation: Automatically correlate local events with external threat intelligence
- Interactive Web Interface: Real-time updates via WebSocket
- Comprehensive Alerts: Combined alerts from local and external sources
- Risk Scoring: Intelligent threat scoring based on multiple factors
- Historical Analysis: Track and analyze security events over time
- Database-backed Reports: PostgreSQL storage with JSONB support
| Feature | Screenshot |
|---|---|
| Login Page |  |
| Create Account |  |
| Feature | Screenshot |
|---|---|
| Main Dashboard |  |
| Advanced Dashboard |  |
| Feature | Screenshot |
|---|---|
| IP Search Interface |  |
| Threat Intelligence |  |
| MITRE ATT&CK Integration |  |
| Feature | Screenshot |
|---|---|
| Security News Feed |  |
| Reports View |  |
| Feature | Screenshot |
|---|---|
| Network Monitoring Dashboard |  |
- Python 3.9 or higher
- PostgreSQL database
- API Keys for:
- AbuseIPDB - IP reputation data
- VirusTotal - Malware intelligence
- Shodan - Network intelligence
- HttpBL - DNS blacklist (optional)
git clone https://github.com/yourusername/threat_intel_aggregator.git
cd threat_intel_aggregatorpython -m venv venv
source venv/bin/activatepip install -r requirements.txtCreate a .env file in the project root:
# API Keys
ABUSEIPDB_API_KEY=your_abuseipdb_key_here
VIRUSTOTAL_API_KEY=your_virustotal_key_here
SHODAN_API_KEY=your_shodan_key_here
HTTPBL_ACCESS_KEY=your_httpbl_key_here
# Flask Configuration
FLASK_SECRET_KEY=your_secret_key_here
FLASK_ENV=development
# Database Configuration
DATABASE_URL=postgresql://username:password@localhost/threat_intel_db# Create PostgreSQL database
createdb threat_intel_db
# Run database migrations
python add_reports_table.pypython run.py- Open your browser and navigate to
http://localhost:5000 - For new users: Click "Register" to create an account
- For existing users: Log in with your credentials
- Default admin account:
- Username:
admin - Password:
admin123
- Username:
New users can create accounts through the web interface:
- Click "Register" in the navigation bar
- Fill out the registration form with:
- Username (3-20 characters, alphanumeric with underscore/dash)
- Email address
- Strong password (8+ chars, uppercase, lowercase, number, special char)
- Password confirmation
- Terms of service agreement
- Click "Create Account"
- Log in with your new credentials
Use the command-line tool for user administration:
# List all users
python manage_users.py list
# Create a new user
python manage_users.py create username email@example.com Password123!
# Delete a user
python manage_users.py delete username
# Reset user password
python manage_users.py reset-password username NewPassword123!
# Show user details
python manage_users.py show username- Password Requirements: Minimum 8 characters with uppercase, lowercase, number, and special character
- Username Validation: 3-20 characters, alphanumeric with underscore/dash
- Email Validation: Proper email format required
- Duplicate Prevention: Username and email must be unique
- Terms Agreement: Users must agree to Terms of Service and Privacy Policy
- Admin Protection: Default admin account cannot be deleted
python test_integration.pythreat_intel_aggregator/
├── 📂 api_clients/ # Threat intelligence API clients
│ ├── abuseipdb_client.py
│ ├── virustotal_client.py
│ ├── shodan_client.py
│ └── base.py
├── 📂 network_monitoring/ # Network monitoring components
│ ├── connection_monitor.py
│ ├── process_monitor.py
│ ├── threat_detector.py
│ ├── yara_scanner.py
│ └── 📂 yara_rules/ # YARA malware detection rules
├── 📂 integration/ # Integration layer
│ ├── event_correlator.py
│ ├── unified_alerter.py
│ └── network_monitor_manager.py
├── 📂 web_dashboard/ # Web application
│ ├── 📂 templates/ # HTML templates
│ ├── 📂 static/ # Static files (CSS, JS)
│ ├── models.py # Database models
│ └── routes.py # Route handlers
├── 📂 threat_intelligence/ # MITRE ATT&CK integration
├── 📂 reports/ # Generated reports
├── 📂 cache/ # Cached threat intelligence data
├── run.py # Web server entry point
├── requirements.txt # Project dependencies
└── README.md # This file
The application integrates with four major threat intelligence APIs:
| API | Purpose | Features |
|---|---|---|
| AbuseIPDB | IP Reputation | IP reputation checking, abuse confidence scoring, country/ISP info |
| VirusTotal | Malware Detection | Malware detection, network activity analysis, community reputation |
| Shodan | Network Intelligence | Port scanning, service detection, vulnerability assessment |
| HttpBL | DNS Blacklist | DNS-based blacklist checking, threat scoring, visitor classification |
Configure alert sensitivity in the integration components:
- Network connection threat score threshold
- Process risk level thresholds
- YARA match thresholds
- Correlation score thresholds
- Connection monitoring interval
- Process monitoring frequency
- YARA scan frequency
- Cache TTL settings
- Connection Tracking: Monitor all TCP/UDP connections
- Process Analysis: Track processes and their network activity
- YARA Scanning: Local malware detection using YARA rules
- Behavioral Analysis: Detect suspicious process behavior
- Automatic Enrichment: Correlate local events with external threat intelligence
- Risk Scoring: Calculate comprehensive threat scores
- Alert Generation: Generate alerts based on correlation results
- Cache Management: Efficient caching of threat intelligence results
- Local Processing: Network monitoring runs locally
- Secure Storage: Encrypted storage of sensitive data
- API Rate Limiting: Respect API rate limits
- Cache Management: Automatic cache expiration
| Issue | Solution |
|---|---|
| Import Errors | Ensure all dependencies are installed: pip install -r requirements.txt |
| API Errors | Verify API keys are correct and have sufficient quota |
| Permission Errors | Network monitoring may require elevated privileges |
| Database Errors | Check PostgreSQL connection and run migrations |
Enable debug logging:
import logging
logging.basicConfig(level=logging.DEBUG)We welcome contributions! Please follow these steps:
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
- Follow PEP 8 style guidelines
- Add tests for new features
- Update documentation as needed
- Ensure all tests pass before submitting
This project is licensed under the MIT License - see the LICENSE file for details.
- AbuseIPDB for IP reputation data
- VirusTotal for malware intelligence
- Shodan for network intelligence
- HttpBL for DNS blacklist data
- YARA for malware detection rules
- Flask for the web framework
- MITRE ATT&CK for threat intelligence framework
If you encounter any issues or have questions:
⭐ Star this repository if you find it useful!