Arduino 8-bit Spectrograph project update

 

Project Update

Arduino linear CCD array for spectrograph.
TCD1304AP Linear Array CCD

This is an update to the Arduino Linear CCD project. It is a simpler circuit, eliminating an op-amp, and more linear because of it. There is a new schematic, PCB, and along with that a new CAD package. I'm using KiCAD now to be a little more standard.

An Arduino can't really keep up with a CCD using the internal analog to digital converter, but with an external 8-bit half-flash ADC it's no problem at all. The ADC is the TI ADC0820, which can digitize up to 666k samples per second. The sample rate here works out to be a hair over 222kSPS using the code below. The 3694 pixel frame takes around 16.66mS to read and digitize, and around 4 seconds to download at 115.2k baud.

Hardware

Microcontroller

The Arduino Uno doesn't have enough RAM for a 3694 pixel buffer. That leaves either the Arduino Mega2560 with just 4kB or a standalone ATmega1284 with 16kB RAM. It wouldn't be easy to embed an Arduino Mega2560, so I went with the ATmega1284P. You don't need the "P" part. There is no power control in this project.

Analog to Digital Converter

The project still uses the ADC0820, but this time it is the ADC0820BCWMX, which is the same as the fastest ADC0820N, and has better linearity, but comes in an SOIC-20 package. For breadboarding, you can still use the PDIP package, or you could get a 20-pin Schmartboard or equivalent. You can find the PDIP parts on ebay, but they are becoming obsolete, so the price will undoubtedly go up.

The ADC0820 ADC has + and - reference inputs designed to set the range of the ADC, so you can focus on the interesting portion of the input range, keeping full 8-bit resolution. The ADC0820BCWMX part has ±1/2 LSB linearity. The ADC0820CCWM has ±1 LSB linearity.

Linear Array CCD

The CCD is a Toshiba TCD1304AP 3648 pixel linear CCD sensor which requires only a single supply voltage. The sensor is driven directly by the microcontroller, and the analog output is buffered by a transistor. The MCLK signal is 888.88kHz, and the pixels are read at 222.22kHz. That's all a limitation of the microcontroller - not the CCD.

The TCD1304AP CCD is available on ebay or aliexpress for anywhere from $4 to $40. Mouser has new stock (TCD1304DG) for $25.

KiCAD Project Files

The KiCAD project requires you to import the CCD and ADC data from a library. The library is included in the project directory. To do that from the project, click the Preferences->Manage Symbol Libraries... and then click on Project Specific Libraries and add one library. The nickname must be "8-bit-spect-1284-rescue", and the path is the path to the "8-bit-spect-1284-rescue.lib" file. That will add the two parts. If the library already exists in your project, click on the path folder icon on the dialog and set the correct path.

Schematic Diagram

Arduino schematic linear CCD array for spectrograph.

Description

A timer on the ATmega1284 generates an 888.88kHz squarewave to be used as the MCLK signal. The code that toggles the clocks on the CCD and ADC does so in cadence with the MCLK signal. The clock edges are aligned by an adjustment of the timer one cycle before the line is read. MCLK runs continuously. The SH, ICG, WR, and RD are only asserted when the CCD is being read.

The output of the CCD is buffered by a PNP transistor before being sent to the ADC. The previous version used an OpAmp to invert and amplify the CCD output to the full range of the ADC. This proved to be unnecessary and added another adjustment. The ADC range is configurable, and it is a simple matter to subtract the pixel value from 255 to invert it.

The original firmware read the CCD continuously and just dumped the buffer when you asked for it. This version does not do that. It reads the buffer on demand. That freed up the main loop to do other things. The firmware reads the CCD 4 times to clear it before exposing, and 99% of the time that is enough. If it has been sitting idle for more than a few minutes with light hitting it, the electrons will flood over the surface and reading it four times will not be enough. If you take a reading and see nothing but a vertical line on the readout, that is probably what happened. Just read it again to clear it. If it still doesn't clear, try reducing the exposure time.

ICSP Connector and JP1

At the pin 1 end of the microcontroller is a programming connector. Beside pin 3 is a jumper marked JP1. The programming connector is for burning the bootloader into the ATmega1284. Jumper JP1, when installed, passes programmer power to the circuit to allow programming with no other power source. When the JP1 is removed, the programmer is isolated. That is how it should normally be to keep from having collisions between the on-board 5V and the programmer's 5V.

Serial Connector

There is a serial connector that matches the FTDI pinout. It is used for uploading the firmware and for sending commands and downloading the video data. The board is marked with DTR on one end of the connector and GND on the other. If you hook it up backwards it shouldn't do too much damage. The pins that would be dangerous are not connected. Make the connector female if you have a male connector on your USB to TTL converter.

Other Parts

Diode D1 is there to protect the circuit should someone accidentally hook the power up backwards. Diode D2 is there for when you disconnect the power. It keeps the regulator from becoming reverse biased. The switch marked RST is the reset on the ATmega1284. Just in case you are working on code and it hangs on you. It's better than disconnecting the power and reconnecting.

Assembled Unit

Arduino assembled 8-bit spectrograph.
Assembled spectrograph PCB

Assembly

Assembly is straightforward, but be aware that the CCD requires a 22 pin socket with a width of 400 mils (10.16mm). You can use female machined pin headers, which I did. If you are using the PCB, I've found that soldering the ADC on first makes life easier. Use the "8-bit-spect-asy.pdf" drawing as a key for installing components. If you are going to make a PC board, the Gerber files can be used at about any PCB shop.

The Bill of Materials (CSV file).

If you are wiring it up on a proto board, use the ADC0820BCN if you can find it. That is the better of the PDIP parts. Be careful to keep the digital and analog wiring away from each other where possible. Use 0.1µF bypass capacitors at every Vcc connection. Keep all of the digital wires as short as practical to keep them from radiating more than necessary. Be especially careful when routing the MCLK signal near analog areas.

I had a little noise that you can see in the Calibration section below. The circuit is sensitive to high frequency power supply noise. That means the little switching power supplies are probably out. You'll want to use a linear power supply rather than a switcher. They are available from Jameco. 12V will work, but 9V is perfect. Or you could run it from a pair of Lithium-ion batteries in series. It's not a low-power circuit, though, at 45mA average. You can tell the linear supplies because they have inputs of 120V 60Hz, rather than 100-240V 50-60Hz. Jameco actually says linear in the description of their linear supplies, but then other places say linear even on their switchers. I guess what I'm trying to say is don't look for a linear wall-wart on ebay.

When I switched to a linear supply, the noise almost completely disappeared and I get essentially flat lines for minimum exposures. At longer exposures the dark signal masks any power supply noise.

Software

Use the MightyCore boards manager plugin by MCUdude. The board will be ATmega1284 and the variant will be the 1284 or the 1284P, depending on which one you have. Pinout is "standard" and the frequency is 16MHz. You can add the MightyCore boards to the boards manager by adding the following url to the "Additional Boards Manager URLs" textbox in Arduino preferences:

https://mcudude.github.io/MightyCore/package_MCUdude_MightyCore_index.json

Then just open the Boards Manager under Tools->Board->Boards Manager and go to the bottom to install the MightyCore boards. The selection of boards will increase considerably to include all of the ATmegaxx4 chips, ATmega16/32, and ATmega8535. Additional programmers will appear, too, and you should use one of them with the MightyCore bootloader.

You will need an AVR programmer to burn the Arduino bootloader into the ATmega1284. The board uses a 10-pin ICSP connector.

The source code for the linear CCD application is very carefully crafted in places to maintain the phase relationship between all of the CCD clocks and stay true to the specification. It takes a logic analyzer to measure the results of any code changes in the "readLine()" function.

#include <util/delay_basic.h>

// ADC RD signal pin 17
#define RD 0x08

// ADC write signal pin 18
#define WR 0x10

// CCD Shift Gate pin 19
#define SH 0x20

// CCD Integration Clear Gate pin 20
#define ICG 0x40

// CCD Master clock pin 21
#define MCLK 0x80

// CCD and ADC clocks
#define CLOCKS PORTD
#define CLOCKP PIND

// ADC data
#define ADATA PINC

#define PIXEL_COUNT 3694

uint8_t pixBuf[PIXEL_COUNT];
char cmdBuffer[16];
int cmdIndex;
int cmdRecvd = 0;
int exposureTime = 10;
bool looping = false;

void setup() {
  // Initialize the clocks.
  DDRD |= (WR | SH | ICG | MCLK | RD);  // Set the clock lines to outputs
  CLOCKS = (ICG + RD + WR);       // Set the ICG, RD, and WR high.

  // Setup the ADC data port as inputs.
//  DDRC = 0;
  
  // Enable the serial port.
  Serial.begin(115200);

  // Setup timer2 to generate an 888.8kHz frequency on pin 21
  TCCR2A =  + (0 << COM2A1) | (1 << COM2A0) | (1 << WGM21) | (0 << WGM20);
  TCCR2B = (0 << WGM22) | (1 << CS20);
  OCR2A = 8;
  TCNT2 = 1;
}

void readLine() {
  
  // Get a pointer to the buffer.
  uint8_t *buf = pixBuf;
  int x = 0;
  uint8_t scratch = 0;
  
  // Disable interrupts or the timer will get us.
  cli();
  
  // Synchronize with MCLK and
  // set ICG low and SH high.
  scratch = CLOCKS;
  scratch &= ~ICG;
  scratch |= SH;
  while(!(CLOCKP & MCLK));
  while((CLOCKP & MCLK));
  TCNT2 = 0;
  _delay_loop_1(1);
  __asm__("nop");
  __asm__("nop");
  __asm__("nop");
  __asm__("nop");
  __asm__("nop");
  CLOCKS = scratch;
  
  // Wait the remainder of 4.5uS @ 16MHz.
  _delay_loop_1(22);
  __asm__("nop");
  __asm__("nop");

  // Set SH low.
  CLOCKS ^= SH;

  // Wait the remainder of 4.5uS.
  _delay_loop_1(23);

  // Start the readout loop at the first pixel.
  CLOCKS |= (RD + WR + ICG + SH);
  
  do {
    // Wait a minimum of 250nS for acquisition.
    _delay_loop_1(2);

    // Start the conversion.
    CLOCKS &= ~WR;
    _delay_loop_1(3);
    CLOCKS |= WR;

    // Wait the rest of 2uS for conversion.
    _delay_loop_1(9);

    // Read the result.
    CLOCKS &= ~RD;

    _delay_loop_1(3);
    __asm__("nop");
    __asm__("nop");
    *buf++ = ADATA;

    // Set the clocks back to idle state
    CLOCKS |= RD;

    // Toggle SH for the next pixel.
    CLOCKS ^= SH;

  } while (++x < PIXEL_COUNT);
  CLOCKS = (ICG + RD + WR);
  sei();
}

void sendData(void) {
  
  int x;

  for (x = 30; x < 3678; ++x) {
    
    Serial.print(x - 30);
    Serial.print(",");
    Serial.print(255-pixBuf[x]);
    Serial.print("\n");
  }
}


void loop() {
  
  int x;
  char ch;
  
  if (cmdRecvd) {
    if (cmdBuffer[0] == 'r') {
      
      // Clear any residual charge.      
      readLine();
      readLine();
      readLine();
      readLine();
      
      // Expose.
      delay(exposureTime);

      // Read and send data.
      readLine();
      sendData();
    
    } else if (cmdBuffer[0] == 'e') {
      String tmp = String(&cmdBuffer[1]);
      exposureTime = tmp.toInt();
      Serial.println(exposureTime);
      
    } else if (cmdBuffer[0] == 'l') {
      looping = !looping;
    }
    memset(cmdBuffer, 0, sizeof(cmdBuffer));
    cmdRecvd = 0;
    cmdIndex = 0;
  }
  if (looping) {
      readLine();
      readLine();
      delay(exposureTime);
      readLine();
  }
  while (Serial.available() > 0) {
    
    ch = Serial.read();
    
    if (ch != 0x0a) {
      cmdBuffer[cmdIndex++] = ch;
    } else {
      cmdRecvd = 1;
    }
  }
}
 

Timing

Arduino linear CCD array output.

At time 0s:0ms:0µs you can see the MCLK is stretched a little. That is to get the ICG, SH, WR and RD lines in sync with the timer, which generates the 888kHz MCLK signal. The code has to maintain this timing relationship throughout the entire read process, reading one pixel every four MCLK cycles. The readLine() function is riddled with assembly language "NOP" code to keep the timing precise. Keep in mind that adding even a single machine instruction requires re-factoring the read timing.

The Sampler Program

The Sampler program is written for Python 3. You must also install pyserial pip install pyserial.

A few lines up from the end of the python code is a line:


    app = Application(master=root, port="/dev/tty.usbserial-00000000", exposure=50)
			

It should match your serial port. If you are using Windows, change that to something like:


    app = Application(master=root, port="COM3", exposure=50)
			

Again, make it match the serial port you are using. When you run the program it draws a graph of the CCD output, but also makes a file in the program directory named 'ccd.csv'. That file can be imported into a spreadsheet application for further processing.

The horizontal resolution of the sensor, and the data returned, is three times the resolution of the graph. The cvs file is written at full resolution. The pixels are binned into 3-pixel wide buckets for the graph.

The buttons are:

  • Sample - Read the CCD and put the trace on the screen.
  • Clear - Clear all traces from the screen.
  • Quit - End the program (you need to do this to keep from generating an error on exit).
  • Samples - How many scans to average together (removes a little noise).
  • Exposure - An integer value from 1 to 1000. The milliseconds to expose.
  • Baseline - Makes a zero second exposure to subtract from other exposures.

When you click on Sample, it draws the graph. It does not erase the previous line. The Clear button does that. If you have an exeptional build and don't have any annoying noise, the Samples radio buttons won't help. Baseline is handy to remove the fixed signal from the video. Exposure is a bit redundant in this version. It sets the exposure time, but doesn't use it. When you click Sample, it pulls the exposure time from the text box and sends it to the spectrograph before requesting a read.

If you ever get what looks like a saturated signal, just redo the exposure. CCDs gather light whether you read them or not, so leaving it alone for 30 minutes is like taking a 30 minute exposure. The code reads the frame 4 times before exposing, but sometimes that isn't enough to get all of the electrons out of the system.

Calibration

To calibrate, start by setting the RV1 pot, "Vref (+)", all of the way clockwise. Set RV2, "Vref (-)" all of the way counter-clockwise. This sets the black level at 5V and the white level at 0V. Get a piece of black electrical tape long enough to cover the ccd + 1/4 inch on each end. Cut a slit across the tape, but not all of the way - just in the center. Use an X-acto knife or similar. Then cover the CCD with the electrical tape in a room with dimmed lighting. The tape will come off without damaging the CCD. You should see a spike when you make a sample. If not, increase the exposure time. If the spike looks more like a square pulse, decrease the exposure time. If you rub your finger down the length of the tape on the CCD, starting at the slit, it will open the slit wider, if you need it. Rubbing your finger toward the slit will tend to close it.

Adjust the exposure time until you get a maximum linear output from the spike. When the spike stops linearly increasing in height, stop increasing the exposure time. In the following image, you can see the curves getting linearly higher in amplitude, then going non-linear on the last step. I started at 100mS and worked up by 50mS per step to 250mS. 200mS was the last linear improvement, so I chose that as the working expoure time for the calibration procedure. The non-linearity is due to sensor saturation.

Clear and take a single sample at the current settings.

Adjust "Vref (+)" counter-clockwise, taking samples along the way, until the bottom line of the signal just reaches about to the bottom of the graph. It may take several turns of the pot to get there. If you overshoot, turn the pot the other direction to bring it back up.

Clear and take a single sample at the current settings.

Turn "Vref (-)" clockwise, taking samples along the way, until the tip of the peak just touches the top line. If you overshoot, you can bring it back down. It will bring the bottom up a bit, but we will tune that out later.

Clear and take a single sample at the current settings.

Go back to the "Vref (+)" pot and turn it counter-clockwise to bring the bottom back down.

Clear and take a single sample at the current settings.

Finally, if needed, back to the "Vref (-)" pot to pull the top up again. I didn't need it in this case, but only because I lucked out and hit it right the first time. If you want to get closer, keep repeating these last two steps until you are satisfied. The amplitude is not calibrated, but it is linear, if the last linear exposure time was chosen above. What this calibration does is increase the digital resolution to use as many of the 256 steps as possible.

You must adjust the pots in that order because, as you saw above, they interact with each other. That is by design. The maximum voltage you can put on Vref(-) is the voltage on Vref(+), and the minimum is ground, so changing Vref(+) changes the range and the current value of Vref(-).

When you remove the tape, clean the CCD glass with alcohol to get any residue off of it.

From now on, you use the exposure time to increase or decrease the range of the signal. If the line goes off the top of the graph, lower the exposure time, and if it falls short, increase the exposure time. If 1000mS isn't long enough, you can change it in the Python Sampler and in the Arduino IDE (both) to be whatever you need it to be. Noise increases with exposure time. If you need it to be shorter, we may need to experiment with microsecond timers. Send me some info on what you are doing via the Contact Form and I'll see what can be done.

Parts Availability

Some of the parts might be a little hard to come by, and I do have some spares. If you are interested in purchasing a component, request it in the Contact Form and I will get back to you on availability. I can say now that I have PCBs for $10 including shipping and a few ADCs at $8 including shipping. The ADCs are the BCWMX parts with better linearity, but I have some CCWM parts coming that have slightly lower linearity (±1 LSB), and a lower price. I'm afraid I can only easily ship to the USA, and may be able to get something to Europe in a pinch, but the shipping will be extra.

Mechanics

Here is where we all go our separate ways. If you want a Raman spectrometer, you will be adding a laser, a grating, and a few mirrors. If you want a barcode reader, you will add a lens. My particular use for this was to characterize dichroic color separation filters used in CCD astrophotography. To do that, I needed a collimated light source, a stage for the filter, a slit, a transmission grating, and the board. I used a 500 lines per mm Edmund Optics holographic grating in a 2" x 2" slide card.

Arduino Board Logo

 

Arduino-Board is the go-to source for information on many available Arduino and Arduino-like boards, tutorials and projects.

Help and Support

Arduino-Board

Stay updated

Sign up if you would like to receive our once monthly newsletter.