interrosec
All articles
Cloud

Hybrid Cloud Network Monitoring: On-Prem Meets Azure

InterroSec Team5 min read

The majority of enterprise production environments today are hybrid: some workloads run on-premises in owned data centers, others run in Azure (or AWS, or GCP), and traffic flows between them over ExpressRoute circuits, VPN gateways, or the public internet. Each environment has its own telemetry systems, its own identity model, and its own operational tooling.

For network security and operations teams, this creates a significant challenge: how do you get a unified view of network behavior when the data is scattered across fundamentally different infrastructures?

The Telemetry Gap in Hybrid Environments

On-premises environments generate rich network telemetry from the infrastructure devices themselves. Cisco, Juniper, and Arista routers and switches export NetFlow and IPFIX records that describe every conversation crossing the device. This telemetry is under your control, flows to your collector, and can be retained and analyzed on your terms.

Azure environments work differently. Traffic between Azure VMs doesn't cross a traditional network device that you control — it flows through Azure's software-defined networking infrastructure. The telemetry you get is what Azure provides: Network Security Group (NSG) Flow Logs, which record accepted and denied flows at the NSG boundary.

The gap is meaningful:

  • Coverage granularity: On-premises NetFlow is per-conversation; NSG Flow Logs are per-NSG rule evaluation. The data models are different enough that naive normalization produces misleading results.
  • Timing: NSG Flow Logs have non-trivial delivery latency (typically 5-10 minutes, sometimes longer). On-premises flow records can be delivered in near real-time (60-second active timeouts).
  • Internal traffic visibility: Traffic between VMs in the same Azure subnet may not cross an NSG, depending on how the NSGs are attached. This creates blind spots for intra-subnet east-west traffic.
  • IP address stability: Azure VMs have ephemeral IPs that can change when VMs are deallocated and reallocated. On-premises IPs are typically stable. This creates correlation challenges when trying to track the same workload across time.

Building a Unified Monitoring Architecture

Despite these differences, it's possible to build unified network monitoring across hybrid environments. The key is normalization: transforming both on-premises flow data and cloud telemetry into a common schema before analysis.

Schema Normalization

A normalized flow record in a hybrid monitoring system should include:

  • Source identity (IP + hostname/instance-ID/FQDN where available)
  • Destination identity
  • Protocol and ports
  • Byte and packet counts
  • Timestamp (normalized to UTC)
  • Traffic origin context (on-premises, Azure-VNet, Azure-external)
  • Workload tag (environment, application, tier — from your tagging schema)

The workload tag is particularly important in Azure, where IP addresses are unreliable long-term identifiers. By tagging Azure resources and propagating those tags into the flow record, you can build baselines and apply segmentation policy based on workload identity rather than ephemeral IP.

Azure-Specific Collection

Enabling NSG Flow Logs requires configuring them explicitly on each NSG (or via Azure Policy for enforcement at scale):

# Enable NSG flow logs via Azure CLI
az network watcher flow-log create \
  --resource-group MyResourceGroup \
  --nsg MyNSG \
  --location eastus \
  --workspace MyLogAnalyticsWorkspace \
  --traffic-analytics true \
  --retention 90 \
  --storage-account mystorageaccount \
  --enabled true \
  --format JSON \
  --log-version 2

NSG Flow Logs version 2 include MAC address information and can be linked to VM names, which helps address the IP stability problem.

For richer internal visibility, Azure Network Watcher also supports packet capture and connection monitoring — though these are more operationally intensive than flow-based monitoring.

Cross-Environment Correlation

Once telemetry from both environments is in the same normalized schema, the real analytical value begins:

Workload communication maps that span both environments. A web tier in Azure talking to a database on-premises is visible in both the Azure NSG Flow Log (the Azure side of the connection) and the on-premises flow data (the data center side). Correlating these gives you the complete picture.

Anomaly baselines that account for hybrid patterns. A traffic baseline that treats Azure and on-premises as separate silos will flag normal hybrid communication as anomalous. A unified baseline models the expected cross-environment traffic as part of normal behavior.

Segmentation policy validation across boundaries. After implementing segmentation policies in both environments, the unified flow view verifies that the policy produces the intended communication pattern across the full hybrid topology.

Azure ExpressRoute and VPN Gateway Monitoring

Traffic crossing the hybrid boundary — via ExpressRoute or VPN Gateway — is particularly important to monitor. These circuits carry sensitive production traffic and are high-value targets for interception or manipulation.

Azure provides connection-level metrics (throughput, packet drops, BGP route counts) for ExpressRoute and VPN Gateway, but not flow-level data. The flow data for hybrid traffic comes from:

  • On-premises edge router: NetFlow from the circuit termination point covers the on-premises perspective of all hybrid traffic.
  • Azure virtual network gateway logs: Log Analytics integration provides connection events and error conditions.
  • NSG Flow Logs on subnets connected to the gateway: Covers Azure-side flow data for hybrid traffic entering Azure VNets.

FlowSight normalizes on-premises NetFlow and Azure NSG Flow Logs into a unified flow store, mapping Azure VM instance IDs to their associated flow records. The result is a single-pane visibility layer that spans both environments — with anomaly detection, topology mapping, and segmentation validation that works across the hybrid boundary, not just within each environment separately.

The Operational Reality

Hybrid cloud network monitoring requires ongoing maintenance as the environment evolves. Azure resources are created and destroyed dynamically; on-premises infrastructure changes more slowly but still changes. The monitoring architecture needs to accommodate both rhythms.

The most practical approach is to lean on automation: tag enforcement via Azure Policy, automatic NSG flow log enablement on new NSGs, and regular topology reconciliation to ensure all new resources are visible. The goal is a monitoring posture where new resources are automatically covered without manual intervention for each deployment.

Unified visibility across hybrid environments isn't a nice-to-have. In environments where incidents begin in one domain and spread to another, monitoring that stops at the boundary is monitoring that misses the attack.

FlowSight

See how FlowSight detects anomalies — get a demo

30-minute walkthrough, no commitment. We'll show you live detection on your network traffic.

Get a Demo