For a long time I worked almost exclusively with input/output streams in java. However a few years ago I started exploring async I/O as well (e.g. running on jboss netty).

However when working in the async I/O world, I could not reuse any of my sync utilities. I also had to start on a rather large library of new I/O utilities and wasn’t sure whether to aim for blocking or non-blocking.

As things go…I wrote an API on top of the existing I/O which supports both blocking and non-blocking and the utilities are written against that.

However because I/O is such a pervasive thing, I am unsure whether it is a wise thing to do. As an example take a Base64 encoder. Yes apache has a sync I/O version and yes java 8 finally has an async encoder. But I needed an encoder that could do both.

Same for quoted-printable, deflate,… and of course it should work for sockets (regular & ssl), files,… in single-threaded and multithreaded situations,… I needed utilities to chain together I/O sources, use delimiter-based or fixed-length I/O wrappers,… I needed protocol handlers for http, ftp,…

Many of these problems can be expressed in a blocking-neutral fashion or -given enough thought- could support both async and sync, allowing wide reusability.

In short: is it wise to introduce an abstraction on top of I/O, forcing basically every library that uses it (directly or indirectly) to use it as well?

UPDATE

At the core are the ReadableByteContainer and WritableByteContainer that -to the discerning eye- will look like a cross between stream I/O and channel I/O.

public interface ReadableByteContainer extends Closeable {
    public int read(byte [] bytes);
    public int read(byte [] bytes, int offset, int length);
}

public interface WritableByteContainer extends Closeable {
    public int write(byte [] bytes);
    public int write(byte [] bytes, int offset, int length);
    public void flush();
}

One of the utils built on it is the “Transcoder” api:

public interface ByteTranscoder {
    public void transcode(ReadableByteContainer in, WritableByteContainer out);
    public void flush(WritableByteContainer out);
}

which allows you to do something like:

ByteContainer container = IOUtils.newByteContainer();
container = IOUtils.wrap(
    container,
    TranscoderUtils.wrapOutput(container, new Base64Encoder())
);
container.write("test".getBytes());
System.out.println(new String(IOUtils.toBytes(container)));

This code will output the base64 encoded bytes of “test”.

The thing is, the Base64Encoder is capable of encoding the data in an asynchronous manner, it won’t block if the output container reports that no data was written nor will it block if the input container reports that no data is available. You can continue the transcoding by calling “transcode()” again.

This works both in a jboss netty context where you get the content piece by asynchronous piece and in the streaming world where everything blocks until it can read/write.