I’m creating a Terraform module to deploy an Azure Log Analytics workspace, a data collection endpoint and a data collection rule.
Now the first two are working just fine, the last one is giving me issues.
The destination I’m using is my Log Analytics workspace, that seems to be fine.
For both the Stream_declaration and data flows, I’m using one called “Custom-Logs_CL”.
The problem seems to be situated somewhere around that.
This is the error I’m getting:
│ Error: creating Data Collection Rule (Subscription: "subscriptionID"
│ Resource Group Name: "rg-phr-loganalytics-dev-weu"
│ Data Collection Rule Name: "pharos-cpi-rule"): datacollectionrules.DataCollectionRulesClient#Create: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPayload" Message="Data collection rule is invalid" Details=[{"code":"InvalidOutputTable","message":"Table for output stream 'Custom-Logs_CL' is not available for destination 'workspaceID'.","target":"properties.dataFlows[0]"},{"code":"InvalidOutputTable","message":"Custom table for stream 'Custom-Logs_CL' is not available for destination 'workspaceID'.","target":"properties.dataFlows[0]"}]
│ with module.log_analytics["pharos-cpi"].azurerm_monitor_data_collection_rule.rule,
│ on moduleslog_analyticsmain.tf line 16, in resource "azurerm_monitor_data_collection_rule" "rule":
│ 16: resource "azurerm_monitor_data_collection_rule" "rule" {
│ creating Data Collection Rule (Subscription:
│ Resource Group Name: "rg-phr-loganalytics-dev-weu"
│ Data Collection Rule Name: "pharos-cpi-rule"):
│ datacollectionrules.DataCollectionRulesClient#Create: Failure responding to
│ request: StatusCode=400 -- Original Error: autorest/azure: Service returned
│ an error. Status=400 Code="InvalidPayload" Message="Data collection rule is
│ invalid" Details=[{"code":"InvalidOutputTable","message":"Table for output
│ stream 'Custom-Logs_CL' is not available for destination
│ 'e877bd70-2fca-429f-a7d4-183fbfe7a044'.","target":"properties.dataFlows[0]"},{"code":"InvalidOutputTable","message":"Custom
│ table for stream 'Custom-Logs_CL' is not available for destination
│ 'SubscriptionID'.","target":"properties.dataFlows[0]"}]
<code>
│ Error: creating Data Collection Rule (Subscription: "subscriptionID"
│ Resource Group Name: "rg-phr-loganalytics-dev-weu"
│ Data Collection Rule Name: "pharos-cpi-rule"): datacollectionrules.DataCollectionRulesClient#Create: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPayload" Message="Data collection rule is invalid" Details=[{"code":"InvalidOutputTable","message":"Table for output stream 'Custom-Logs_CL' is not available for destination 'workspaceID'.","target":"properties.dataFlows[0]"},{"code":"InvalidOutputTable","message":"Custom table for stream 'Custom-Logs_CL' is not available for destination 'workspaceID'.","target":"properties.dataFlows[0]"}]
│
│ with module.log_analytics["pharos-cpi"].azurerm_monitor_data_collection_rule.rule,
│ on moduleslog_analyticsmain.tf line 16, in resource "azurerm_monitor_data_collection_rule" "rule":
│ 16: resource "azurerm_monitor_data_collection_rule" "rule" {
│
│ creating Data Collection Rule (Subscription:
│ "Subscriptionid"
│ Resource Group Name: "rg-phr-loganalytics-dev-weu"
│ Data Collection Rule Name: "pharos-cpi-rule"):
│ datacollectionrules.DataCollectionRulesClient#Create: Failure responding to
│ request: StatusCode=400 -- Original Error: autorest/azure: Service returned
│ an error. Status=400 Code="InvalidPayload" Message="Data collection rule is
│ invalid" Details=[{"code":"InvalidOutputTable","message":"Table for output
│ stream 'Custom-Logs_CL' is not available for destination
│ 'e877bd70-2fca-429f-a7d4-183fbfe7a044'.","target":"properties.dataFlows[0]"},{"code":"InvalidOutputTable","message":"Custom
│ table for stream 'Custom-Logs_CL' is not available for destination
│ 'SubscriptionID'.","target":"properties.dataFlows[0]"}]
</code>
│ Error: creating Data Collection Rule (Subscription: "subscriptionID"
│ Resource Group Name: "rg-phr-loganalytics-dev-weu"
│ Data Collection Rule Name: "pharos-cpi-rule"): datacollectionrules.DataCollectionRulesClient#Create: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPayload" Message="Data collection rule is invalid" Details=[{"code":"InvalidOutputTable","message":"Table for output stream 'Custom-Logs_CL' is not available for destination 'workspaceID'.","target":"properties.dataFlows[0]"},{"code":"InvalidOutputTable","message":"Custom table for stream 'Custom-Logs_CL' is not available for destination 'workspaceID'.","target":"properties.dataFlows[0]"}]
│
│ with module.log_analytics["pharos-cpi"].azurerm_monitor_data_collection_rule.rule,
│ on moduleslog_analyticsmain.tf line 16, in resource "azurerm_monitor_data_collection_rule" "rule":
│ 16: resource "azurerm_monitor_data_collection_rule" "rule" {
│
│ creating Data Collection Rule (Subscription:
│ "Subscriptionid"
│ Resource Group Name: "rg-phr-loganalytics-dev-weu"
│ Data Collection Rule Name: "pharos-cpi-rule"):
│ datacollectionrules.DataCollectionRulesClient#Create: Failure responding to
│ request: StatusCode=400 -- Original Error: autorest/azure: Service returned
│ an error. Status=400 Code="InvalidPayload" Message="Data collection rule is
│ invalid" Details=[{"code":"InvalidOutputTable","message":"Table for output
│ stream 'Custom-Logs_CL' is not available for destination
│ 'e877bd70-2fca-429f-a7d4-183fbfe7a044'.","target":"properties.dataFlows[0]"},{"code":"InvalidOutputTable","message":"Custom
│ table for stream 'Custom-Logs_CL' is not available for destination
│ 'SubscriptionID'.","target":"properties.dataFlows[0]"}]
To me it sounds that it’s complaining about the Log Analytics workspace table that does not exist yet, so, I tried adding a Terraform block for this. I used this command, but as the documentation states; “This resource does not create or destroy tables. “
So a two headed question, is this table not existing really my issue, and if so, is there another way to deploy it (using Terraform)?
I don’t think it’s added value, but if someone would need my code to check something, I’m happy to paste some snippets, let me know!
Thanks!