Although this seems like a trivial task I ran into a big obstacle which was primarily caused by a lack in my knowledge of how generics in Java work.
Because all constructors are either deprecated or package private the builder pattern is the way to go.
So I simply tried this (assisted by the fact that IntelliJ 'suggested' this):
Path outputPath = new Path("/home/nbasjes/tmp/measurement.parquet");
ParquetWriter<Measurement> parquetWriter =
AvroParquetWriter<Measurement>.builder(outputPath)
.withSchema(Measurement.getClassSchema())
.withCompressionCodec(CompressionCodecName.GZIP)
.build();
None of the unit tests in the Parquet code base use the builder pattern yet; they are still on the methods that have been deprecated.
Looking at the source of the class that needs to do the work you'll see this:
Looking at the source of the class that needs to do the work you'll see this:
public class AvroParquetWriter<T> extends ParquetWriter<T> { public static <T> Builder<T> builder(Path file) { return new Builder<T>(file); }
After talking to a colleague of mine he pointed out that calling a static method in generic has a slightly different syntax. Apparently calling such a method requires the following syntax (which works like a charm):
Path outputPath = new Path("/home/nbasjes/tmp/measurement.parquet");
ParquetWriter<Measurement> parquetWriter =
AvroParquetWriter.<Measurement>builder(outputPath)
.withSchema(Measurement.getClassSchema())
.withCompressionCodec(CompressionCodecName.GZIP)
.build();
Let me guess; You don't see the difference?
It is the '.' before the call to the static builder method.
So instead of this
AvroParquetWriter<Measurement>.builder(outputPath)
you must do this
AvroParquetWriter.<Measurement>builder(outputPath)