Grafana has been deployed inside EKS, The DataSource is added via below ConfigMap but it is not authorized to connect to CloudWatch:
<code>apiVersion: v1
kind: ConfigMap
metadata:
name: datasources
namespace: monitoring
data:
datasources.yaml: |
apiVersion: 1
datasources:
- access: proxy
name: CloudWatch
type: cloudwatch
jsonData:
defaultRegion: ca-central-1
authType: default
assumeRoleArn: arn:aws:iam::****:role/EKS-Grafana-Role
</code>
<code>apiVersion: v1
kind: ConfigMap
metadata:
name: datasources
namespace: monitoring
data:
datasources.yaml: |
apiVersion: 1
datasources:
- access: proxy
name: CloudWatch
type: cloudwatch
jsonData:
defaultRegion: ca-central-1
authType: default
assumeRoleArn: arn:aws:iam::****:role/EKS-Grafana-Role
</code>
apiVersion: v1
kind: ConfigMap
metadata:
name: datasources
namespace: monitoring
data:
datasources.yaml: |
apiVersion: 1
datasources:
- access: proxy
name: CloudWatch
type: cloudwatch
jsonData:
defaultRegion: ca-central-1
authType: default
assumeRoleArn: arn:aws:iam::****:role/EKS-Grafana-Role
IAM role binded to serviceAccount is defined as below, to test it I have added the CloudWatch full access as well
<code>**Policies:
**
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"*",
"arn:aws:iam::597572605549:role/EKS-Grafana-Role"
]
}
]
}
**TrustRelationship
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::*****:oidc-provider/oidc.eks.ca-central-1.amazonaws.com/id/******"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.ca-central-1.amazonaws.com/id/******:sub": "system:serviceaccount:monitoring:grafana-service-account"
}
}
}
]
}
</code>
<code>**Policies:
**
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"*",
"arn:aws:iam::597572605549:role/EKS-Grafana-Role"
]
}
]
}
**TrustRelationship
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::*****:oidc-provider/oidc.eks.ca-central-1.amazonaws.com/id/******"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.ca-central-1.amazonaws.com/id/******:sub": "system:serviceaccount:monitoring:grafana-service-account"
}
}
}
]
}
</code>
**Policies:
**
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"*",
"arn:aws:iam::597572605549:role/EKS-Grafana-Role"
]
}
]
}
**TrustRelationship
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::*****:oidc-provider/oidc.eks.ca-central-1.amazonaws.com/id/******"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.ca-central-1.amazonaws.com/id/******:sub": "system:serviceaccount:monitoring:grafana-service-account"
}
}
}
]
}
Now the issue is :
<code>1. CloudWatch metrics query failed: AccessDenied: User: arn:aws:sts::***:assumed-role/EKS-Grafana-Role/1723494657709439590 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::***:role/EKS-Grafana-Role status code: 403, request id: ed6588cd-b67a-41eb-9fe6-617f83ebf371 2. CloudWatch logs query failed: AccessDenied: User: arn:aws:sts::***:assumed-role/EKS-Grafana-Role/1723494657709439590 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::****:role/EKS-Grafana-Role status code: 403, request id: 27533cf5-54c0-45de-bec3-dc9beb2e550e```
I have tested the solution with adding CloudWatch as dataSourrce manually
</code>
<code>1. CloudWatch metrics query failed: AccessDenied: User: arn:aws:sts::***:assumed-role/EKS-Grafana-Role/1723494657709439590 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::***:role/EKS-Grafana-Role status code: 403, request id: ed6588cd-b67a-41eb-9fe6-617f83ebf371 2. CloudWatch logs query failed: AccessDenied: User: arn:aws:sts::***:assumed-role/EKS-Grafana-Role/1723494657709439590 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::****:role/EKS-Grafana-Role status code: 403, request id: 27533cf5-54c0-45de-bec3-dc9beb2e550e```
I have tested the solution with adding CloudWatch as dataSourrce manually
</code>
1. CloudWatch metrics query failed: AccessDenied: User: arn:aws:sts::***:assumed-role/EKS-Grafana-Role/1723494657709439590 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::***:role/EKS-Grafana-Role status code: 403, request id: ed6588cd-b67a-41eb-9fe6-617f83ebf371 2. CloudWatch logs query failed: AccessDenied: User: arn:aws:sts::***:assumed-role/EKS-Grafana-Role/1723494657709439590 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::****:role/EKS-Grafana-Role status code: 403, request id: 27533cf5-54c0-45de-bec3-dc9beb2e550e```
I have tested the solution with adding CloudWatch as dataSourrce manually