I want to programmatically create indexed bitmaps in formats other than PixelFormat.Format8bppIndexed and need help in understanding why my current implementation is causing heap corruption.
I’ve already established that it’s possible to populate the bitmap from an array which uses 8 bits to represent each pixel, and this approach seems to work well, however it’s not very memory efficient. I’d like to understand how to use a source array which uses the same number of bits per pixel as the destination bitmap uses, in order to be able to work with large images without using excessive amounts of memory.
My code
In order to reduce confusion between the number of bits per pixel in the source array and the number of bits per pixel in the bitmap, I have this enumeration
/// <summary>
/// Enumeration of possible numbers of bits per pixel in the source data used to populate a bitmap.
/// </summary>
public enum DataFormat
{
/// <summary>
/// One bit per pixel.
/// </summary>
Format1bpp = 1,
/// <summary>
/// Four bits per pixel.
/// </summary>
Format4bpp = 4,
/// <summary>
/// Eight bits per pixel.
/// </summary>
Format8bpp = 8,
}
This is the updated version of the IndexedBitmapHelper class from my previous question
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Text;
using System.Transactions;
/// <summary>
/// Helper class for working with indexed bitmaps.
/// </summary>
public static class IndexedBitmapHelper
{
/// <summary>
/// Sets the palette and pixels of an indexed Bitmap.
/// </summary>
/// <param name="destinationBitmap">The Bitmap to populate.</param>
/// <param name="colourTable">
/// An array of the possible colours that the Bitmap's pixels can be set to.
/// </param>
/// <param name="colourIndexes">
/// Each entry in this array represents one pixel and is an index into the <paramref name="colourTable"/>.
/// </param>
/// <param name="dataFormat">
/// Determines the number of bits used in the <paramref name="colourIndexes"/> to represent a single pixel.
/// This determines the <see cref="PixelFormat"/> to use when locking the bitmap.
/// </param>
public static void Populate(
Bitmap destinationBitmap,
Color[] colourTable,
byte[] colourIndexes,
DataFormat dataFormat)
{
SetPalette(destinationBitmap, colourTable);
SetPixels(destinationBitmap, colourIndexes, dataFormat);
}
private static void SetPalette(Bitmap destinationBitmap, Color[] colourTable)
{
var numberOfColours = colourTable.Length;
// The Palette property is of type ColorPalette, which doesn't have a public constructor
// because that would allow people to change the number of colours in a Bitmap's palette.
// So instead of creating a new ColorPalette, we need to take a copy of the one created
// by the Bitmap constructor.
var copyOfPalette = destinationBitmap.Palette;
for (var i = 0; i < numberOfColours; i++)
{
copyOfPalette.Entries[i] = colourTable[i];
}
destinationBitmap.Palette = copyOfPalette;
}
private static void SetPixels(Bitmap destinationBitmap, byte[] colourIndexes, DataFormat dataFormat)
{
//using var trans = new TransactionScope();
var width = destinationBitmap.Width;
var height = destinationBitmap.Height;
var dataPixelFormat = GetPixelFormat(dataFormat);
var bitmapData = destinationBitmap.LockBits(
new Rectangle(0, 0, width, height),
ImageLockMode.WriteOnly,
dataPixelFormat);
var dataOffset = 0; // pointer into the memory occupied by the byte array of colour indexes
var scanPtr = bitmapData.Scan0.ToInt64(); // pointer into the memory occupied by the bitmap
var scanPtrIncrement = bitmapData.Stride;
var arrayBytesPerRow = BppCalculator.GetArrayBytesPerRow(width, dataFormat);
try
{
for (var y = 0; y < height; ++y)
{
// Copy one row of pixels from the colour indexes to the destination bitmap
Marshal.Copy(colourIndexes, dataOffset, new IntPtr(scanPtr), width);
dataOffset += arrayBytesPerRow;
scanPtr += scanPtrIncrement;
}
}
finally
{
destinationBitmap.UnlockBits(bitmapData);
}
}
/// <summary>
/// Gets the PixelFormat to be passed to the Bitmap.LockBits method when the array data
/// is in the supplied DataFormat.
/// </summary>
/// <param name="dataFormat">The number of pixels representing a single pixel in the array data.</param>
/// <returns>The PixelFormat which corresponds to the supplied PixelFormat.</returns>
private static PixelFormat GetPixelFormat(DataFormat dataFormat)
{
var pixelFormat = dataFormat switch
{
DataFormat.Format1bpp => PixelFormat.Format1bppIndexed,
DataFormat.Format4bpp => PixelFormat.Format4bppIndexed,
DataFormat.Format8bpp => PixelFormat.Format8bppIndexed,
_ => throw new ArgumentException($"Unexpected data format: {dataFormat}", nameof(dataFormat)),
};
return pixelFormat;
}
And a little helper class, just to reduce the size of the IndexedBitmapHelper class
/// <summary>
/// Performs calculations around the relationships between image sizes, array sizes
/// and numbers of bits per pixel in array data.
/// </summary>
public static class BppCalculator
{
/// <summary>
/// Gets the number of pixels represented by a byte.
/// </summary>
/// <param name="dataFormat">
/// The number of bits representing a single pixel in the array data.
/// </param>
/// <returns>The divisor.</returns>
public static int GetPixelsPerByte(DataFormat dataFormat)
{
var pixelsPerByte = dataFormat switch
{
DataFormat.Format8bpp => 1,
DataFormat.Format4bpp => 2,
DataFormat.Format1bpp => 8,
_ => throw new ArgumentException($"Unexpected data format: {dataFormat}", nameof(dataFormat)),
};
return pixelsPerByte;
}
/// <summary>
/// Gets the number of bytes required in array data to represent one row of pixels
/// in an indexed bitmap.
/// </summary>
/// <param name="widthInPixels">Width of the bitmap in pixels.</param>
/// <param name="dataFormat">Number of bits per pixel in the array data.</param>
/// <returns>The number of bytes required to represent one row of pixels.</returns>
public static int GetArrayBytesPerRow(int widthInPixels, DataFormat dataFormat)
{
var bitsPerRow = widthInPixels * (int)dataFormat;
// Round up to the next whole byte
var mod8 = bitsPerRow % 8;
if (mod8 > 0)
{
bitsPerRow += 8 - (bitsPerRow % 8);
}
// Convert from bits to bytes
return bitsPerRow / 8;
}
}
My expectation
I ought to be able to use the above code to create, for example, a 2×2 pixel 1bpp bitmap containing a checkerboard pattern, like this
var destinationBitmap = new Bitmap(2, 2, PixelFormat.Format1bppIndexed);
var colourTable = new Color[] { Colour.Black, Colour.White };
var colourIndexes = new byte[]
{
0x01000000,
0x10000000,
};
IndexedBitmapHelper.Populate(bitmap, colourTable, colourIndexes, DataFormat.Format1bpp);
(The reason for the extra 6 bits per row is that a row of pixels must be represented by a whole number of bytes, even if the number of bits needed to represent the colours of that row isn’t a multiple of 8)
However, this results in a runtime error
System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values
at this line
Marshal.Copy(colourIndexes, dataOffset, new IntPtr(scanPtr), width);
I found that appending another byte to the pixelIndexes array…
var colourIndexes = new byte[]
{
0x01000000,
0x10000000,
0,
};
… stops the runtime error, and the image looks correct.
Here’s another example, this time a 10×12 1bpp image:
var destinationBitmap = new Bitmap(10, 12, PixelFormat.Format1bppIndexed);
var colourTable = new Color[] { Colour.Black, Colour.White };
var colourIndexes = new byte[]
{
// Expected colour indexes:
// 0 0 0 0 0 1 1 1 1 1
// 0 1 1 1 1 0 0 0 0 1
// 0 1 1 1 1 0 0 0 0 1
// 0 1 1 1 1 0 0 0 0 1
// 0 1 1 1 1 0 0 0 0 1
// 1 0 1 0 0 1 1 0 1 0
// 1 0 1 0 0 1 1 0 1 0
// 1 0 1 0 0 1 1 0 1 0
// 1 0 1 1 1 0 0 0 1 0
// 1 0 0 0 0 1 1 1 1 0
// 1 1 1 1 1 0 0 0 0 0
// Last meaningful bit per row is here, remaining bits are padding to the next whole byte
// |
0b00000111, 0b11000000,
0b01111000, 0b01000000,
0b01111000, 0b01000000,
0b01111000, 0b01000000,
0b01111000, 0b01000000,
0b01111000, 0b01000000,
0b10100110, 0b10000000,
0b10100110, 0b10000000,
0b10100110, 0b10000000,
0b10111000, 0b10000000,
0b10000111, 0b10000000,
0b11111000, 0b00000000, // last row of image
0, 0, 0, 0, 0, 0, 0, 0, // padding to avoid ArgumentOutOfRangeException in Marshal.Copy
};
IndexedBitmapHelper.Populate(bitmap, colourTable, colourIndexes, DataFormat.Format1bpp);
This also appears to create the correct image, however if I remove any of the last 7 bytes, I again get the ArgumentOutOfRangeException.
So why do I need these extra bytes at the end of my source array? How do I know how many extra bytes I need for a given image?
What I’ve tried
To empirically capture the number of bytes needed for a given image, in the hope of being able to see a pattern, I came up with the following approach:
- Initialise the colourIndexes array to the size I think it needs to be based on the image size and bits per pixel
- Attempt to populate the bitmap using this array
- If an
ArgumentOutOfRangeExceptionis thrown, repeat from step 1, but with one more byte in the array - If no exception is thrown, write the array size to the console
I tried the above for a number of different image widths and heights using this class
using System.Drawing;
using System.Drawing.Imaging;
/// <summary>
/// Class for calculating the smallest size of colourIndexes array which is needed
/// to create an indexed bitmap of the supplied size using the supplied DataFormat.
/// </summary>
public class ArraySizeCalculator
{
public void CalculateAll(DataFormat dataFormat)
{
var primes = new int[] { 1, 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 51, 53, 57, 59 };
foreach (var height in primes)
{
foreach (var width in primes.Where(p => p >= height))
{
_ = this.Calculate(width, height, dataFormat);
}
}
}
public int Calculate(int width, int height, DataFormat dataFormat)
{
var arraySize = width * height;
Color[] colourTable;
PixelFormat bitmapFormat;
switch (dataFormat)
{
case DataFormat.Format1bpp:
arraySize /= 8;
bitmapFormat = PixelFormat.Format1bppIndexed;
colourTable = new Color[2];
break;
case DataFormat.Format4bpp:
arraySize /= 2;
bitmapFormat = PixelFormat.Format4bppIndexed;
colourTable = new Color[16];
break;
case DataFormat.Format8bpp:
bitmapFormat = PixelFormat.Format8bppIndexed;
colourTable = new Color[256];
break;
default:
throw new ArgumentException(dataFormat.ToString(), nameof(dataFormat));
}
Array.Fill(colourTable, Color.DeepSkyBlue);
while (true)
{
var bitmap = new Bitmap(width, height, bitmapFormat);
try
{
//Console.WriteLine($"Trying array size {arraySize}");
var colourIndexes = new byte[arraySize];
Array.Fill<byte>(colourIndexes, 0);
IndexedBitmapHelper.Populate(bitmap, colourTable, colourIndexes, dataFormat, skipValidation: true);
break;
}
catch (ArgumentOutOfRangeException)
{
arraySize++;
//Console.WriteLine($"Argument out of range, trying array size {arraySize}");
}
}
Console.WriteLine($"{dataFormat} {width}x{height}: {arraySize}");
return arraySize;
}
}
What happened
Despite wrapping the call to the CalculateAll method in a try { ... } catch (Exception ex) { ... } construct, and debugging it from Visual Studio as a console application, after iterating through a number of different image sizes, the program crashes and a dialogue box is displayed with the message
A problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available.
I.e. neither the Visual Studio debugger nor the catch blocks in my code have caught the exception. The number of iterations before the program crashes changes from one run to the next, and if I change the program to just call the Calculate method for the image size being calculated when the crash occurs, the program doesn’t crash.
The only clue to what went wrong is an Error event in the Windows Application event log
Faulting application name: ConsoleApp1.exe, version: 1.0.0.0, time stamp: 0x65f90000
Faulting module name: ntdll.dll, version: 10.0.14393.6343, time stamp: 0x6502749b
Exception code: 0xc0000374
Fault offset: 0x000d8f91
Faulting process id: 0x330
Unexplained crashes related to ntdll.dll tells me that ntdll.dll isn’t actually the problem and that the important piece of information here is exception code 0xc0000374, which means heap corruption has occurred, quite possibly long before the program actually crashed.
Heap corruption is a much more difficult problem to solve. The corruption can occur many millions of instructions before a crash occurs. Worse, erroneous code could alter memory that it doesn’t own, such as changing the account balance of your bank account accidentally and not causing a crash.
I understand that by using Bitmap.LockBits and Marshal.Copy, I’m effectively bypassing the normal .net managed code approach and instead writing directly to the memory occupied by the bitmap object. And probably also writing to some memory occupied by something else, which is clearly a Bad Thing. This makes me worry that even when I appear to have successfully created a bitmap, I may also have corrupted some other memory unrelated to the bitmap.
So…. to sum things up
Is there something wrong with the way the IndexedBitmapHelper.PopulatePixels method writes to the bitmap’s area of memory?
Is there a way to guard against writing to memory which is not part of the bitmap?
Why are the extra bytes needed in the pixelIndexes array, how can I tell how many are needed, and what should they contain?