I want to process a sort of “mixed format” file using two different writers (this question is related to my earlier question about how to write such a file: Writing into the same file with two `using` statements and two writers)
My problem is that when I open a file and then create a StreamReader to read it, every other reader I create on the file gets an empty stream and cannot read anything.
The minimum example is as follows. Let’s say we have a file containing two lines, the first saying first line and the second saying second line.
Then, writing the following:
using (FileStream file = new(fileName, FileMode.Open, FileAccess.Read))
{
List<string> strings = new();
using (StreamReader reader = new StreamReader(file, leaveOpen: true))
{
strings.Add(reader.ReadLine());
}
using (StreamReader reader = new StreamReader(file, leaveOpen: true))
{
strings.Add(reader.ReadLine());
}
}
I would expect the list strings to contain the elements "first line" and "second line". Instead, when I run the code, I get the first element "first line", but the second element of strings is null. I am obviously not understanding something here, but I don’t know what.
35
StreamReader will read a big chunk of the file at a time (otherwise it would have to read character-by-character to find the line endings, which would be horribly inefficient). You therefore can’t rely on it leaving the FileStream’s position at any particular point.
OP has clarified that they’re reading NRRD-formatted data. This comprises a list of headers, separated from binary data by two newlines:
NRRD000X
<field>: <desc>
<field>: <desc>
# <comment>
...
<field>: <desc>
<key>:=<value>
<key>:=<value>
<key>:=<value>
# <comment>
<data><data><data><data><data><data>...
(This is similar to reading HTTP headers, followed by a body).
The easiest way to read this is probably to scan through the file looking for the two newlines next to each other. Do bear in mind that newlines in NRRD can be both LF and CRLF.
Once you have that position, you can read the first part of the file as text, then seek back to the start of the binary data and read the rest as binary. This is unfortunately going to be a bit wordy, but that’s probably unavoidable unfortunately.
Alternatively, something like the following seems to work. This uses a StreamReader to handle the messy business of working out what a newline is (LF vs CRLF etc), but constrains it by giving it a MemoryStream to read, which we only add one byte to at a time.
Since StreamReader.ReadLine returns a line when it reaches the end of the stream (as well as when it reads a newline), we also have to check StreamReader.EndOfStream.
var buffer = new MemoryStream();
var reader = new StreamReader(buffer);
byte[] b = new byte[1];
int bufferPosition = 0;
while (true)
{
int bytesRead = file.Read(b, 0, b.Length);
if (bytesRead == 0)
{
// We ran out of file data before reaching the end of the headers
break;
}
buffer.Write(b, 0, b.Length);
buffer.Position = bufferPosition;
reader.DiscardBufferedData();
string? line = reader.ReadLine();
if (line != null && !reader.EndOfStream)
{
Console.WriteLine(line);
// Clear out the MemoryStream. We need to re-append the byte we just read
buffer.SetLength(0);
buffer.Write(b, 0, b.Length);
if (reader.ReadLine() == "")
{
break;
}
}
}
Run it online here.
Another approach is to wrap the FileStream in another stream, which artificially limits the number of bytes which can be read. The following example is very incomplete, but does do the job here:
public class ConstrainedStream : Stream
{
private readonly Stream _baseStream;
public long ConstrainedLength { get; set; }
public override bool CanRead => _baseStream.CanRead;
public override bool CanSeek => _baseStream.CanSeek;
public override bool CanWrite => false;
public override long Length => Math.Min(ConstrainedLength, _baseStream.Length);
public override long Position
{
get => _baseStream.Position;
set => _baseStream.Position = value;
}
public ConstrainedStream(Stream baseStream)
{
_baseStream = baseStream;
}
public override void Flush() => _baseStream.Flush();
public override int Read(byte[] buffer, int offset, int count)
{
long maxLength = Math.Min(Length, ConstrainedLength) - Position;
return _baseStream.Read(buffer, offset, (int)Math.Min(count, maxLength));
}
public override long Seek(long offset, SeekOrigin origin)
{
return _baseStream.Seek(offset, origin);
}
public override void SetLength(long value) => _baseStream.SetLength(value);
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
This lets you write something like this:
var constrainedStream = new ConstrainedStream(file) { ConstrainedLength = 1 };
var reader = new StreamReader(constrainedStream);
long startPosition = 0;
while (true)
{
constrainedStream.ConstrainedLength++;
if (constrainedStream.ConstrainedLength > file.Length)
{
// Run out of file
break;
}
reader.DiscardBufferedData();
file.Position = startPosition;
string? line = reader.ReadLine();
if (line != null && !reader.EndOfStream)
{
Console.WriteLine(line);
startPosition = file.Position - 1;
if (reader.ReadLine() == "")
{
break;
}
}
}
Run it online here.
1