setting up a glue crawler job to read from an s3 bucket and create a glue catalog database. once the resource are created , how can i trigger it. can i hook it to s3 object creation ? also, can crawler detect relationship between the data and create the tables accordingly, similar to tables in a relational database, with foreign key to link between tables ? for example, one file has names of product and other files have product details.
<code>[{ "id" : 1,
"product" : "Tablet",
},
{ "id" : 2,
"product" : "headphones"
}]
</code>
<code>[{ "id" : 1,
"product" : "Tablet",
},
{ "id" : 2,
"product" : "headphones"
}]
</code>
[{ "id" : 1,
"product" : "Tablet",
},
{ "id" : 2,
"product" : "headphones"
}]
file 2.
[{“name”: “Tablet”, “price” : 350 …}, {}]
<code>resource "aws_glue_crawler" "example" {
database_name = aws_glue_catalog_database.example.name
name = "example"
role = aws_iam_role.example.arn
s3_target {
path = "s3://${aws_s3_bucket.example.bucket}"
}
}
</code>
<code>resource "aws_glue_crawler" "example" {
database_name = aws_glue_catalog_database.example.name
name = "example"
role = aws_iam_role.example.arn
s3_target {
path = "s3://${aws_s3_bucket.example.bucket}"
}
}
</code>
resource "aws_glue_crawler" "example" {
database_name = aws_glue_catalog_database.example.name
name = "example"
role = aws_iam_role.example.arn
s3_target {
path = "s3://${aws_s3_bucket.example.bucket}"
}
}