University websites are constantly exposed to the internet, making them attractive targets for automated scanners and attackers. Open Journal Systems (OJS), being one of the most widely used academic publishing platforms, is no exception. Traditional protection mechanisms such as signature-based WAFs work well against known attacks, but they often struggle when facing new payload variations or previously unseen attack patterns.
This project was our attempt to solve that problem by building a real-time anomaly detection pipeline powered by an unsupervised Autoencoder model. Instead of sitting inline with production traffic, the system quietly observes every incoming request, analyzes it in the background, and immediately alerts administrators whenever something suspicious appears. The idea was simple: detect abnormal behavior without introducing additional latency to the actual OJS service.
My Role
I worked as the Backend & Infrastructure Engineer. While the machine learning model was developed by our ML engineer, I was responsible for everything involving the system infrastructure and the backend:
- Designing the overall architecture
- Building the HTTP traffic scanner in Go (Fiber)
- Implementing asynchronous traffic mirroring using Nginx
- Deploying and configuring the virtual machine infrastructure
- Setting up service communication using gRPC
- Integrating Telegram Bot API for instant alerts
System Architecture
The overall architecture follows a mirrored traffic approach.
Instead of inspecting requests inline, Nginx duplicates every incoming request using asynchronous traffic mirroring. The original request continues directly to OJS with virtually no additional delay, while the mirrored copy is forwarded to the backend scanner for analysis. This architecture allows security monitoring to run independently from the production workload.
Keep in mind, we don't use Modsecurity to make it more lighter and real-time, because our goal is to detect the attack (IDS), not to prevent it.
Backend Traffic Scanner
The backend scanner is written in Golang using Fiber. Its responsibility is much more than simply forwarding requests. Every mirrored request is transformed into structured features that can later be consumed by the machine learning model. Depending on the request, the extracted information includes:
- Source IP
- HTTP method
- URI
- Query parameters
- Headers
- Request body
- Content length
- User-Agent
- Various metadata useful for anomaly detection
Once extracted, these features are forwarded to the FastAPI inference service. If the prediction indicates malicious behavior, the scanner stores the result in PostgreSQL and immediately pushes a Telegram notification to the administrator.
Integrating the Machine Learning Service
The Autoencoder model itself runs separately as a FastAPI service. We intentionally kept the ML model isolated from the backend scanner because it makes deployment and maintenance significantly easier. The backend only needs to send structured JSON to the inference endpoint and receive a prediction response. That separation also allows the ML model to be updated independently without touching the rest of the backend.
Infrastructure
One of the biggest parts of this project was wiring everything together. The deployment consisted of several independent services communicating over the internal network:
- Nginx
- OJS
- Golang Backend Scanner
- Python FastAPI
- PostgreSQL
- React Dashboard
I configured the VM environment, networking, reverse proxy rules, and service routing so every component could communicate reliably while remaining loosely coupled. Since mirrored traffic is processed completely outside the production request path, failures in the scanner or ML service never interrupt access to OJS itself. That isolation was one of the main design goals.
Monitoring Dashboard
To make the collected data useful, we built a monitoring dashboard using ReactJS with PostgreSQL as its data source. The dashboard provides:
- Real-time traffic statistics
- Detection history
- Attack classifications
- Risk distribution
- Overall detection metrics
- Feedback System (true/false)
Instead of manually digging through server logs, administrators can quickly see what's happening inside the system. Since we also implement the feedback system, the ML can easily be retrained from the feedback data.
Telegram Alerting
Whenever the backend receives a malicious prediction from the Autoencoder, it immediately sends a structured Telegram notification. The alert includes technical details such as:
- Source IP
- HTTP method
- Target endpoint
- Detection result
- Request information
- Timestamp
This gives administrators immediate visibility without having to continuously monitor the dashboard.
Testing
To evaluate the system, we performed security testing using OWASP ZAP. Various attack payloads were generated through fuzzing and automated scanning, including:
- SQL Injection
- Cross-Site Scripting (XSS)
- Cross-Site Request Forgery (CSRF)
These requests flowed through the same mirrored pipeline as normal traffic, allowing us to validate the end-to-end detection workflow without affecting the production OJS service.
Repository
The repository of this work can be found in this Github organization repository.
Final Thoughts
This project reminded me that building machine learning systems is rarely just about the model.
Most of the engineering effort went into making independent services work together reliably. The Autoencoder was only one piece of the puzzle. The real challenge was designing an architecture where production traffic stayed fast, security analysis happened in real time, and every component could fail independently without bringing down the actual OJS service.
Looking back, this was one of my favorite backend projects because it sat right at the intersection of networking, distributed systems, and security engineering.