Transformers are powerful middleware components that process, enrich, and modify DNS traffic data as it flows through your DNS-collector pipeline. They enable real-time data transformation, filtering, analysis, and privacy protection without requiring external processing tools.
Transformers execute in a specific sequence to ensure data consistency and optimal performance.
Important
The execution order of transformers is currently hardcoded in the Go source and cannot be changed via the configuration file.
The logical processing order is as follows:
- Normalization - Normalize: Standardizes DNS message format.
- Filtering - Traffic Filtering: Applies sampling and filtering rules.
- Enrichment - Elements like GeoIP, ATags, Suspicious Traffic Detector, etc.
- Anonymization - User Privacy: Masking or hashing components.
- Aggregation - Traffic Reducer: Deduplicates repetitive queries (terminal point).
- Ordering - Reordering: Sorts DNS messages (terminal point).
| Transformer | Capabilities | Impact |
|---|---|---|
| Normalize | • Convert domain names to lowercase • Extract TLD and TLD+1 components • Standardize text formatting • Clean malformed queries |
Essential for consistent data analysis and storage |
| Reordering | • Sort DNS messages by timestamp • Handle out-of-order packet processing • Maintain chronological data flow |
Critical for accurate time-series analysis |
| Transformer | Capabilities | Use Cases |
|---|---|---|
| Traffic Filtering | • Downsampling: Reduce data volume by percentage • Domain Filtering: Drop/allow specific domains • IP Filtering: Filter by client or server IP • Response Code Filtering: Filter by DNS response codes |
• High-volume environment optimization • Focused monitoring on specific domains • Compliance and policy enforcement |
| Traffic Reducer | • Detect identical repeated queries • Log unique queries only once • Maintain occurrence counters • Reduce storage requirements |
• Minimize storage costs • Focus on unique DNS patterns • Performance optimization |
| Transformer | Detection Capabilities | Security Benefits |
|---|---|---|
| Suspicious Traffic Detector | • Malformed Packets: Invalid DNS structure • Oversized Queries: Potential DDoS indicators • Uncommon Query Types: Rare or suspicious Qtypes • Invalid Characters: Malicious domain encoding • Excessive Labels: DNS tunneling attempts • Long Domain Names: Covert channel detection |
• Early threat detection • DNS tunneling prevention • Malware C&C identification • DDoS attack mitigation |
| Newly Observed Domains | • Track first-time domain appearances • Identify domain generation algorithms (DGA) • Monitor new subdomain creation • Alert on suspicious registration patterns |
• Zero-day domain detection • Brand protection monitoring • Typosquatting identification • Advanced persistent threat tracking |
| Transformer | Privacy Features | Compliance Support |
|---|---|---|
| User Privacy | • IP Anonymization: Hash or mask client IPs • Domain Minimization: Reduce domain specificity • SHA1 Hashing: Irreversible data protection • Configurable Privacy Levels: Granular control |
• GDPR compliance • Internal privacy policies • Data sharing agreements • Research data anonymization |
| Transformer | Metrics & Analysis | Operational Value |
|---|---|---|
| Latency Computing | • Query-Response Matching: Correlate requests with responses • Round-Trip Time: Measure DNS resolution speed • Timeout Detection: Identify unanswered queries • Performance Trends: Track resolution performance |
• SLA monitoring • Performance troubleshooting • Capacity planning • Service quality assurance |
| Traffic Prediction | • Feature Extraction: ML-ready data preparation • Pattern Recognition: Identify traffic patterns • Anomaly Scoring: Statistical deviation detection • Trend Analysis: Historical comparison |
• Predictive scaling • Anomaly detection • Capacity forecasting • AI/ML model training |
| Transformer | Enrichment Capabilities | Enhanced Insights |
|---|---|---|
| GeoIP Metadata | • Country Identification: Client geolocation • City-Level Data: Detailed location information • ASN Mapping: Internet service provider data • IP Intelligence: Threat reputation scoring |
• Geographic traffic analysis • Compliance monitoring • Threat intelligence correlation • Content delivery optimization |
| Data Extractor | • Base64 Encoding: Full DNS payload preservation • Binary Data Handling: Raw packet analysis • Metadata Extraction: Protocol-level details • Custom Field Addition: Flexible data enhancement |
• Deep packet inspection • Forensic analysis • Custom analytics • Advanced research |
| REST Lookup | • Custom Data Addition: Flexible data enhancement | • Business intelligence integration |
| Transformer | Transformation Features | Integration Benefits |
|---|---|---|
| Additional Tags | • Custom Metadata: Business-specific labels • Conditional Tagging: Rule-based classification • Dynamic Values: Runtime data injection • Multi-Tag Support: Complex categorization |
• Business intelligence integration • Custom analytics dashboards • Automated workflows • Data organization |
| JSON Relabeling | • Field Renaming: Standardize JSON keys • Field Removal: Clean unnecessary data • Structure Modification: Reshape data format • Nested Object Handling: Deep JSON manipulation |
• System integration • Data standardization • Storage optimization • API compatibility |
| DNS Message Rewrite | • Field Value Modification: Change DNS record data • Conditional Rewriting: Rule-based transformations • Pattern Matching: Regex-based modifications • Multi-Field Updates: Bulk data changes |
• Data normalization • Privacy compliance • Testing scenarios • Data migration |