I am stuck on a problem with my Scala SBT (1.7.2) project.
Situation:
I already have working configuration to generate code from *.avsc, *.avdl files via "com.julianpeeters" % "sbt-avrohugger" % "2.0.0-RC19"
plugin. Now, I must add code generation from *.proto files. I have followed instruction of setting up ScalaPB (from here). Well, protobuf generation works as intended, but for some reason the avro code generation is affected – some files are created, but it looks like the plugin didn’t have time to finish up work and just stopped in the middle, making the tests fail in result.
Code:
plugins.sbt:
addSbtPlugin("com.julianpeeters" % "sbt-avrohugger" % "2.0.0-RC19")
addSbtPlugin("com.thesamet" % "sbt-protoc" % "1.0.7")
libraryDependencies += "com.thesamet.scalapb" %% "compilerplugin" % "0.11.15"
build.sbt:
lazy val `job` = (project in file("."))
.settings(avroHuggerSettings)
.settings(protobufSettings)
lazy val avroHuggerSettings = Seq.concat(
inConfig(IntegrationTest)(SbtAvrohugger.baseSettings),
Seq(
Compile / sourceGenerators += (Compile / avroScalaGenerate).taskValue,
Test / sourceGenerators += (Test / avroScalaGenerate).taskValue,
IntegrationTest / sourceGenerators += (IntegrationTest / avroScalaGenerate).taskValue,
Compile / avroSourceDirectories := Seq(new java.io.File(s"${baseDirectory.value}/src/main/resources/avro")),
Test / avroSourceDirectories := Seq(new java.io.File(s"${baseDirectory.value}/src/test/resources/avro")),
IntegrationTest / avroSourceDirectories := Seq(new java.io.File(s"${baseDirectory.value}/src/it/resources/avro")),
Compile / avroScalaCustomTypes := avroCustomTypes,
Test / avroScalaCustomTypes := avroCustomTypes,
IntegrationTest / avroScalaCustomTypes := avroCustomTypes
)
)
lazy val protobufSettings = Seq(
Compile / PB.targets := Seq(
scalapb.gen() -> (Compile / sourceManaged).value
)
My Avro files are located in src/main/resources/avro
, my Proto files are located in src/main/protobuf
.
With such setup my proto files are generated correctly, but only some of the avro files are done. Mind you, both plugins generate code into target/scala-2.12/src_managed/main
, but into different packages.
Output logs snippet (sbt assembly
):
[info] Considering source directories ...resources/avro
[info] Compiling AVSC ... .avsc to .../target/scala-2.12/src_managed/main/compiled_avro
[info] Compiling AVSC ... .avsc to .../target/scala-2.12/src_managed/main/compiled_avro
[info] Compiling Avro IDL ... .avdl
[info] Compiling 4 protobuf files to .../target/scala-2.12/src_managed/main
[info] Compiling Avro IDL ... .avdl
[info] Compiling Avro IDL ... .avdl
[info] Compiling Avro IDL ... .avdl
[info] Compiling Avro IDL ... .avdl
[info] Compiling Avro IDL ... .avdl
[info] Compiling Avro IDL ... .avdl
[info] compiling 139 Scala sources to .../target/scala-2.12/classes
[info] done compiling
//same as above but for some test avro classes
//failing test output
[error] (Test / compileIncremental) Compilation failed
[error] Total time: 9 s, completed Jun 5, 2024, 4:11:10 PM
My attempts to fix:
I am not good with the SBT so it was a trial and error, but I’ve tried to make either plugin run before the other one, with no results. I tried to use the examples from this stack thread, like that:
lazy val generateAvro = taskKey[Unit]("Avro task")
lazy val generateProto = taskKey[Unit]("Proto task")
generateAvro := {
println("Starting avro task ...")
//tried both with Seq() and without, doesn't matter
Seq(
Compile / sourceGenerators += (Compile / avroScalaGenerate).taskValue,
Test / sourceGenerators += (Test / avroScalaGenerate).taskValue,
IntegrationTest / sourceGenerators += (IntegrationTest / avroScalaGenerate).taskValue,
Compile / avroSourceDirectories := Seq(new java.io.File(s"${baseDirectory.value}/src/main/resources/avro")),
Test / avroSourceDirectories := Seq(new java.io.File(s"${baseDirectory.value}/src/test/resources/avro")),
IntegrationTest / avroSourceDirectories := Seq(new java.io.File(s"${baseDirectory.value}/src/it/resources/avro")),
Compile / avroScalaCustomTypes := avroCustomTypes,
Test / avroScalaCustomTypes := avroCustomTypes,
IntegrationTest / avroScalaCustomTypes := avroCustomTypes,
(Compile / avroScalaGenerate).value // tried with and without it
)
println("Avro task ended")
}
generateProto := {
println("Starting proto task ...")
Compile / PB.targets := Seq(
scalapb.gen() -> (Compile / sourceManaged).value
)
println("Proto task ended")
}
Compile / compile := Def.taskDyn {
val result = (Compile / compile).value
Def.task {
val _ = generateProto.value
result
}
}.dependsOn(generateAvro).value
Unfortunately it does not work, I guess that the content of those tasks is merely a config for the plugins, that work independently from my will during the compilation, right?
Please, show me some guidance how could I make those plugins work together in SBT? Making those plugins work in some orchestrated way, change the configuration for any of it, anything to get all the files generate please.