What is logged in Azure and how long are the logs retained?

This article gives an overview of what type of activity is logged into Azure and for what duration of time.

Through activity logs, you can determine:

• what operations were taken on the resources in your subscription
• who started the operation
• when the operation occurred
• the status of the operation
• the values of other properties that might help you research the operation

Activity Logs Provides insight into the operations on each Azure resource in the subscription from the outside (the management plane) in addition to updates on Service Health events. Use the Activity Log, to determine the what, who, and when for any write operations (PUT, POST, DELETE) taken on the resources in your subscription. You can also understand the status of the operation and other relevant properties. There is a single Activity log for each Azure subscription.Activity logs are kept for 90 days. You can query for any range of dates, as long as the starting date isn't more than 90 days in the past. 

NOTE:  
The Azure Activity Log is primarily for activities that occur in Azure Resource Manager. It does not track resources using the Classic/RDFE model. Some Classic resource types have a proxy resource provider in Azure Resource Manager (for example, Microsoft.ClassicCompute). If you interact with a Classic resource type through Azure Resource Manager using these proxy resource providers, the operations appear in the Activity Log. If you interact with a Classic resource type outside of the Azure Resource Manager proxies, your actions are only recorded in the Operation Log. The Operation Log can be browsed in a separate section of the portal.

If a compliance policy requires logs to be kept for longer than 90 days, you must export them.

You can retrieve information from the activity logs through the portal, PowerShell, Azure CLI, Insights REST API, or Insights .NET library.

Important: When exporting logs to event huban  or a storage account, costs are likely to increase.

NOTE: The method for sending the Azure Activity log to Azure Storage and Azure Event Hubs has changed to diagnostic settings. This article describes the legacy method which is in the process of being deprecated. See Update to Collect and analyze Azure Activity log in Azure Monitor for a comparison.

The Azure Activity Log provides insight into subscription-level events that have occurred in your Azure subscription. In addition to viewing the Activity log in the Azure portal or copying it to a Log Analytics workspace where it can be analyzed with other data collected by Azure Monitor, you can create a log profile to archive the Activity log to an Azure storage account or stream it to an Event Hub.

Archiving the Activity Log to a storage account is useful if you would like to retain your log data longer than 90 days (with full control over the retention policy) for audit, static analysis, or backup. If you only need to retain your events for 90 days or less you do not need to set up archival to a storage account, since Activity Log events are retained in the Azure platform for 90 days.

Azure Event Hubs is a data streaming platform and event ingestion service that can receive and process millions of events per second. Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters. Two ways you might use the streaming capability for the Activity Log are:

•     Stream to third-party logging and telemetry systems: Over time, Azure Event Hubs streaming will become the mechanism to pipe your Activity Log into third-party SIEMs and log analytics solutions.
•     Build a custom telemetry and logging platform: If you already have a custom-built telemetry platform or are thinking about building one, the highly scalable publish-subscribe nature of Event Hubs enables you to flexibly ingest the activity log.

Platform logs in Azure, including the Azure Activity log and resource logs, provide detailed diagnostic and auditing information for Azure resources and the Azure platform they depend on. Platform Metrics are collected by default and typically stored in the Azure Monitor metrics database.

Each Azure resource requires its own diagnostic setting, which defines the following criteria:

• Categories of logs and metric data sent to the destinations defined in the setting. The available categories will vary for different resource types.
• One or more destinations to send the logs. Current destinations include Log Analytics workspace, Event Hubs, and Azure Storage.

A single diagnostic setting can define no more than one of each of the destinations. If you want to send data to more than one of a particular destination type (for example, two different Log Analytics workspaces), then create multiple settings. Each resource can have up to 5 diagnostic settings.

Destinations

Platform logs and metrics can be sent to the destinations in the following table. Follow each link in the following table for details on sending data to that destination.

Create diagnostic settings in Azure portal

You can configure diagnostic settings in the Azure portal either from the Azure Monitor menu or from the menu for the resource.

1. Where you configure diagnostic settings in the Azure portal depends on the resource.

   • For a single resource, click Diagnostic settings under Monitor in the resource's menu.

     

   • For one or more resources, click Diagnostic settings under Settings in the Azure Monitor menu and then click on the resource.

     

   • For the Activity log, click Activity log in the Azure Monitor menu and then Diagnostic settings. Make sure you disable any legacy configuration for the Activity log.

     

2. If no settings exist on the resource you have selected, you are prompted to create a setting. Click Add diagnostic setting.

   

   If there are existing settings on the resource, you see a list of settings already configured. Either click Add diagnostic setting to add a new setting or Edit setting to edit an existing one. Each setting can have no more than one of each of the destination types.

   

3. Give your setting a name if it doesn't already have one.

   

4. Category details (what to route) - Check the box for each category of data you want to send to destinations specified later. The list of categories varies for each Azure service.

   • AllMetrics routes a resource's platform metrics into the Azure Logs store, but in log form. These metrics are usually sent only to the Azure Monitor metrics time-series database. Sending them to the Azure Monitor Logs store (which is searchable via Log Analytics) you to integrate them into queries which search across other logs. This option may not be available for all resource types. When it is supported, Azure Monitor supported metrics lists what metrics are collected for what resource types.

     Note:

     Sending multi-dimensional metrics via diagnostic settings is not currently supported. Metrics with dimensions are exported as flattened single dimensional metrics, aggregated across dimension values.

     For example: The 'IOReadBytes' metric on an Blockchain can be explored and charted on a per node level. However, when exported via diagnostic settings, the metric exported represents as all read bytes for all nodes.

   • Logs lists the different categories available depending on the resource type. Check any categories that you would like to route to a destination.

5. Destination details - Check the box for each destination. When you check each box, options appear to allow you to add additional information.

   

   1. Log Analytics - Enter the subscription and workspace. If you don't have a workspace, you need to create one before proceeding.

   2. Event hubs - Specify the following criteria:

      • The subscription which the event hub is part of
      • The Event hub namespace - If you do not yet have one, you'll need to create one
      • An Event hub name (optional) to send all data to. If you don't specify a name, an event hub is created for each log category. If you are sending multiple categories, you may want to specify a name to limit the number of event hubs created.
      • An Event Hub policy (optional) A policy defines the permissions that the streaming mechanism has. For more information, see

   3. Storage - Choose the subscription, storage account, and retention policy.

      

      Tip:

      Consider setting the retention policy to 0 and manually deleting your data from storage using a scheduled job to avoid possible confusion in the future.

      First, if you are using storage for archiving, you generally want your data around for more than 365 days. Second, if you choose a retention policy that is greater than 0, the expiration date is attached to the logs at the time of storage. You can't change the date for those logs once stored.

      For example, if you set the retention policy for WorkflowRuntime to 180 days and then 24 hours later set it to 365 days, the logs stored during those first 24 hours will be automatically deleted after 180 days, while all subsequent logs of that type will be automatically deleted after 365 days. Changing the retention policy later doesn't make the first 24 hours of logs stay around for 365 days.

6. Click Save.

After a few moments, the new setting appears in your list of settings for this resource, and logs are streamed to the specified destinations as new event data is generated. It may take up to 15 minutes between when an event is emitted and when it appears in a Log Analytics workspace.