I created a partitioning logic on my code to parse a file into more files and download one partition of all of them, i merged it into the api, it dont partitioning i have readed all my code and i dont see the problem, i dont know if its on the download call or on the select file code
i want some help, i will provide some of my code
public Single<FileResponse> Descarga(int pageSize, String sortField, String sortOrder) {
return Single.defer(() -> {
final var fileName = LocalDate.now ()+compress (sortField+sortOrder+pageSize) +".json.gz";
final var path = UUID.randomUUID () + "-" + fileName;
final var fileOutputStream = new FileOutputStream ( path );
final var gzipOutputStream = new GZIPOutputStream ( fileOutputStream );
final var mapper = DatabindCodec.mapper ();
final var jsonGenerator = mapper.getFactory ().createGenerator ( gzipOutputStream, JsonEncoding.UTF8 );
jsonGenerator.writeStartArray ();
log.info ( "Selected pageSize {}", pageSize );
log.info ( "sorted by {}", sortField );
return select ( pageSize, sortField, sortOrder )
.toFlowable ()
.map ( row -> row.getString ( 0 ) )
.doOnNext ( node->mapper.writeValue ( jsonGenerator,node ) )
.flatMapSingle ( ds->service.prtRant ( JsonArray.of ( ds ) ) )
.ignoreElements ()
.doOnTerminate ( ()->{
jsonGenerator.writeEndArray ();
jsonGenerator.close ();
gzipOutputStream.close ();
fileOutputStream.close ();
} )
.toSingleDefault ( FileResponse.builder ()
.path ( path )
.fileName ( fileName )
.build ());
});
}
and this is my select code
public Single<JsonArray> select(int pageSize, String sortField, String sortOrder) {
final var statement = QueryBuilder.selectFrom ( "marketplace", "bankgateway_transaction" )
.json ()
.all ()
.build ()
.setPageSize ( pageSize );
final var fetch= cqlSessionSingle.flatMap ( cqlSession -> Single.fromFuture ( cqlSession.executeAsync ( statement ).toCompletableFuture ()))
.map(AsyncPagingIterable::currentPage)
.flatMapObservable(Observable::fromIterable)
.map(row -> row.getString(0))
.toList();
final var partitioning = fetch.flatMap ( data->service.prtRant ( JsonArray.of ( data ) ) ).map(JsonArray::new);
return partitioning;
}
one make a call to a cassandra DB and then transform it into list then partitioned and then into a JsonArray to make it more easy to manage on the web, then the download method is call
and this is the partitioning method
private Single<List<Long>> pTinter(Long count) {
final var parZ = cPZ(count);
final var tPar = cTP(count, parZ);
return Single.zip(tPar,parZ, (total, size) -> {
//Implementacion de la logica de particion
List<Long> PartitionId = new ArrayList<>();
for (long i = 0; i < total; i++) {
PartitionId.add(i*size);
}
// Ordenar las particiones según el criterio de clasificación proporcionado
return new ArrayList<> ( PartitionId );
});
}
Like it look, i partitioned a JsonArray and then write the content on a fileOutput wich gonna be hashed and has a .json.gz type, im expecting the partitioning and get a file less than the original, but im getting a file much bigger