Times they are a changin'

Here's a dandy chart showing the decrease in frame rate as the WFS image size is increased. Each sub-aperture is around 22 px across, and each pixel is around 6 um, so that gives you an idea of the beam size necessary. I also varied the shutter speed, but this should obviously be set to the minimum value necessary to see the centroids.



Aligning the beam is turning out to be problematic because the divergence is significant as the beam size shrinks. The good news is that a windfall in our equipment budget means I'm finally getting a HeNe laser like I always wanted. expanding it is slightly more involved than with the diode laser I'm using now, but I should be able to clean it up using a spatial filter and get a pretty nice spot.

As a matter of fact, we also ordered a new DM to use in place of the retarded SLM I've struggled with. The new DM contains around 60 actuators compared to the 31 in the current model. It might seem odd to use the higher density DM as a disturbance generator, but in RL atmospheric turbulence is always going to generate higher (spatial) frequency distortion than can be corrected with a finite actuator DM.

This basically means that all the active components of my experiment are going to be repaired/replaced in the next few weeks. I really really hope that's it because frankly i'm tired of dealing with all this optics shit.

A word about speed

Even with the latest improvements its unlikely that I'll every be able to run my experiment fast enough for real real time (need 10's of kHz for that). Still, 50 Hz is better then 3. There are several factors that affect the temporal sampling rate of my experiment. Most of them are interrelated, so it helps to sort out just what impacts what.

Essentially, there are 3 time-comsuming operations that have to be completed each iteration. Ignoring the time to calculate control signals, these are: Commanding the DM, capturing the WFS image, and processing the image to generate the slope vector.

Sending DM Commands:
This is basically fixed around 100 Hz currently. I'm still not sure if the actuators actually reach their steady state positions at that speed, hopefully I'll have that sorted out next week. The unknown here is that the fwrite command used here can time out, basically crashing the whole enchilada.

Capturing WFS image:
As I found out last week, the frame rate of the WFS is a function of the frame size and the shutter speed. The frame size dictates the number of sub-apertures used, and hence determines the spatial sampling frequency. Intuitively, reconstructing (and hence controlling) higher spatial frequencies means a slower frame rate.

Short shutter times also increase the frame rate, but only up to a certain point. Beyond that, the frame size basically determines how fast the thing can go. Generally, the idea is to select the minimum shutter speed necessary to accurately image the sub-apertures.

Computing Slope Vector:
Again, this is a function of how many sub-aperatures are being used. I'm not exactly sure how this varies though since its cumbersome to alter the code every time for a new image size.

Clearly a lot of these parameters are fixed by the experiment. The DM only has 31 actuators, so there's a limit to the required spatial sampling frequency. Whats more, for a given laser the shutter speed can just be set to the smallest value that still allows the centroid algorithm to work. That basically leaves the WFS frame size as the only parameter that can effectively be varied. I'm hoping to write up a little script this weekend to see how the overall frame rate is affected by this.

Rewriting the slope estimation code turned into quite a shit storm of coordinates and indices, so it took me most of Wed. and Thurs. to write it and test everything. To answer the questions from before:

(1) The slope algorithm runs at 100 Hz as written for an array of 10x10 sub-apertures and an image around 220x220px. The image actually needs to be slightly larger since each sub-aperature isn't exactly 22 pixels wide (something I need to fix later). Suspiciously all three control processes: sending commands, reading a WFS image and processing the slope, top out around the same speed. The slope script can be sped up to around 300 Hz if I pass in the file containing the lenselet coordinates rather than load them each iteration (TURBO MODE!).

(2) Currently with a 10x10 sub-aperature WFS image, the whole process runs around 50 Hz. The strange thing is that the speeds aren't consistent with simply adding the speeds for individual processes. For example sending a DM command and just reading an image occurs at around 100 Hz, even though that is approximately the individual speed for each of those. Adding the slope finding script knocks this down to 50 Hz. Using turbo mode on the slopes causes the DM command script to hang (at fwrite).

(3) 50 Hz is probably sufficient for now, and reducing the shutter further doesn't result in much faster frame rates. The issue now is going to be alignment since a 220x220 px square is less than 2mm^2. Pushing to faster frame rates would mean even smaller beam sizes.

(4) Its probably possible to work with the current laser, but we have some equipment money to use so a new laser is probably in the cards.

I'll be out Mon and Tues taking advantage of my strict grad student schedule. The goal for next week is to have the beam resized with the current laser (probably minus the target splitter), and run the classical controller at 50 Hz. If possible I'd also like to look at the actuator dynamics with this faster WFS frame rate.

Word.

I'm an Idiot

Today I finally relented and had a telecon with some of the guys who manufacture our DM and WFS. They basically took over my PC and were able to get high frame rates right away by reducing the shutter speed to small values. I don't know why I didn't do this or do it successfully before now. Let this be a lesson to you kids: even if you're a moron don't give up hope on that phd.

I'm able to get rates >1000 fps if the shutter is small enough and the image size is 10's of px. The problem now is that, for a given image size, the exposure of the WFS image is entirely determined by the required frame rate. Obviously Nyquist's sampling theorem applies to spatial sampling, and increasing the number of lenselets used to measure the beam increases the image size. This means that there's a trade off between the spatial and temporal sampling resolution that I can get for a fixed laser intensity.



Right now it doesn't make much sense to sample much faster than the rate at which I can spit out DM commands (since I'm ignoring actuator dynamics atm), so a frame rate around 120 fps should be good for now. Screwing with the number of lenselets is a hassle since it requires changing the beam size and computing a new reconstruction matrix, but it appears I can get around 120 fps and still have an exposure sufficient to image the spots if I use a grid of around 10x10 sub-apertures (a WFS image of around 220x220 px).

Tomorrow I want to get an idea of what kind of rates I can expect in the finished product to determine what the bottleneck is now. Hopefully it'll be software this time. This will involve:

(1) Adjusting the slope-finding code to work with a 10x10 lenselet array. How fast will it run with this smaller size?

(2) Test out how fast the system can apply a command and return a slope vector -- no need to resize the beam quite yet. Does it matter what part of the frame is used? Prob not with a CMOS cam would be my guess.

(3) If thumbs up on the overall frame rate, figure out what the new beam size needs to be and start on the hardware changes.

(4) Once the beam size is good, determine if the shutter can be sped up at all. Today I had to remove the splitter to the target cam since these new exposures were too short to capture all the centroids. If adding that splitter requires too high a shutter it might be time to think about a class IIIa laser.

Once this is all hunky dory I'll need to know if the actuator dynamics are important at these higher frame rates. Shouldn't be a problem figuring this out using the code I wrote to test the messed up DM.

Finally, time to step off on this shite.

One word: plastics

Its nice to find out, after almost a year of running experiments, that my DM is f0xred and probably has been for quite some time. I was always suspicious since my reconstruction errors were sometimes astronomical, but control always seemed possible nonetheless. I found out the thing was screwed by accident.

Basically, running the DM faster involved using a new command that allowed multiple actuators to be commanded simultaneously, rather than one at a time as before. The command rate of the DM should therefore be ~30x faster, easily into the hundreds of hertz. However for whatever reason using this command on my DM caused havoc, essentially freezing it in the zero position until I manually unplugged it. Days went by as I tried various explanations, before I eventually wrote a script that seemed to work on the other AO experiment we have using the same DM and driver box (I had to make some creative guesses on how to format the parameters used in the command). The code was indeed around 30x faster, but my DM was still crapped out.

Coincidentally, I ran some test using the old command to look at the linearity of the DM surface to voltage commands. As expected, some of the them responded quadratically, but the large majority didn't respond at all. After some back and forth with the manufacturers, I was able to trigger the WFS asynchronously as a command was sent, and was able to actually catch the actuators moving. Sure enough, some seemed fine, but in many cases the influence functions decayed significantly (sometimes to zero) within milliseconds of the command.

[insert pic]

Clearly something was fucked. After another week of test to convince everyone I wasn't hallucinating, we finally got a new DM yesterday. This mirror has some odd annular actuator arrangement, so I doubt we'll end up using it permanently, but so far it seems to be taking the new commands without complaint. I haven't checked for actuator decay yet, but so far there's no evidence of funny business. I also have a voltage sampler to check the response of the drive box, but I doubt that's malfunctioning.

Its irritating though that even after demonstrating what I consider to be a significant amount of code (instead of just using "their software"), the guys who make the mirror still seem to think I'm something of an optical retard. I guess its always better to overachieve.

Tomorrow:
- test the DM for actuator decay
- use the voltage sampler to test the drive box
- examine the runtime errors I get with the manufacturer's software

My advisor's made it clear that speeding up the SLM is a low priority at the moment, and I'm to focus on getting the DM and WFS working faster.

I suppose this makes sense, but accomplishing those things means a lot of waiting around to hear back from the manufacturer. In the mean time I"m going to try my hand at writing some basic adaptive controllers using the same architecture as in the beam pointing experiments. I already have a Simulink model that actually runs the experiment, so instead of screwing around with the SLM, for the mean time I can just add desired disturbance wavefronts to the measurement. From a control perspective this shouldn't make any difference, and it'll let me get my feet wet with some adaptive methods. I can get an identified disturbance model that can spit out disturbances with realistic statistics, so doing this should be "trivial."

Didn't Ben Franklin have syphilis?

As if I didn't have enough going on right now, I'd like to eventually write a complete set of HDR functions that I can actually use with a DSLR. I already have part of this done already for use with AO, but there are still a few features that are left:

1. Image alignment: since I don't really carry a tripod around everywhere, it'd be nice to have an image alignment function that can automatically align several images taken by hand. If you consider the problem as sliding one image around on top of another, this is really an (convex?) optimization problem: find the optimal (x,y) displacement to minimize some alignment metric, like the mean pixel difference or something. I think there are a couple algorithms out there that do this by gradient descent, so I'll have to look those up but the general idea is simple. The problem with HDR is that each image has a different exposure, so somehow that has to be corrected. Maybe weighting each pixel by its intensity of something to ignore saturated areas...

2. Creating a HDR image map: basically already done following the method by Debevec.

3. Tone mapping: there are a boatload of algorithms for this, and I'm still not sure which one's the simplest to implement and yields decent results. Right now the leading candidate is Gradient Domain HDR Compression (Fattal 2002) since the results look pretty good and I understand most of the paper.

Who knows if I'll ever get around to this. I recently read a nice how-to on doing time-lapse videos using a Canon point and shoot, so if I could somehow combine that to make HDR time-lapses it could be epic.

Nope

Last week I finished writing a new slope calculation script based on idea of using 2 linear operators and dividing the result. IS this faster than looping through each sub-aperture in Matlab? Nope.

The problem is that the matrix to calculate the unnormalized slopes is 2N*M, where N is the number of sub-apertures and M is the total number of pixels in the image. Since the WFS has around 1600 sub-apertures, and the image is more than 1000x1000, this matrix is gigantic. Even though its sparse, it takes up more than 1.5 MB of space. The matrix to calculate the total intensity of each sub-aperture is around the same order of magnitude, so combined the script has to load over 2MB of data each time a WFS image is captured. This takes more than a second in Matlab, so that option is off the table for now. Simply loading them into the workspace beforehand and passing them to the script is much faster, but still around the same speed as using loops.

Ultimately, when I get the DM working faster (still waiting for word on that from the manufacturer), I'll have to write the slope algorithm as a mex file and run it in C. What's another wasted week?

Faster

I've heard from the company that manufactures our DM and WFS that they can achieve frame rates an order of magnitude higher than what I'm capable of doing, around 800 Hz. This blows my mind since I have no idea how I make any more than incremental improvements with the Matlab commands I'm using now. In a sense this pisses me off since I'm not sure how we would have found this out if my advisor hadn't brought it up. It seems common in this industry for companies to ignore what most people are doing and write their own, barely adequate, proprietary software, and then go into stealth mode with the support.

Anyway, we're focusing on the DM first since that's most likely the harder problem. I'm actually skeptical they can get the frame rates they claim, but who knows. I've sent them the code I'm current using-code that was adapted from stuff they sent us-so I should hear something soon. I fully expect to hear complaints about a variety of minor coding transgressions and other shit that doesn't matter.

Nonetheless, I'm assuming that I can eventually speed things up significantly on the hardware side, which means that my software will have to change as well. Right now the most significant software bottleneck is the script for calculating the slope vector from the WFS image. For a full frame 1280x1024 Hartmann image, calculating the slopes in Matlab takes around 0.08 seconds for a grid of around 40x40 lenselets. The problem, as I mentioned before, is that finding the centroids is not a linear operation since you have to divide by the sub-aperature intensity, thus you can't just multiply the image by some matrix. Loops in Matlab are shit, so looping over all the sub-aperatures is what causes most of the delay.

Right now I'd like to try to reduce the computation time in Matlab by reducing the centroid calculation to 2 linear operations on the image. if I is the vectorized image, then there is a (sparse, large) matrix A such that S1 = A*I, where S1 is a vector of unnormalized slopes. There is also another matrix Q such that L = Q*I where L contains the total intensity in each sub-aperature. The slope vector is then (in Matlab speak) S = S1./[L;L] (since S1 contains x and y slopes).

I have no idea if this is really faster, but there's enough evidence to give it a try. Coming up with A efficiently though isn't easy, but hopetully I'll finish that tomorrow.

5.19.09

Last weekend I ran a couple long SPGD experiments with and without the SLM to see if it had any noticeable effect on the performance. I ran each lasted for around 10,000 iterations, and since I can only update the SLM every 3 seconds they each took over 8 frickin hours to run. I feel like a biologist.



For the "w/ SLM" case the image was set to a focus with a sinusoidally varying intensity such that the maximum phase distortion occurs at the center of the focus at the peak sinusoidal amplitue. Maybe I'm just cynical, but its not really clear to me that the SLM has any effect. Its nice the the algorithm can sort of maximize the image cost function (J), ignoring those unexplained spikes, but its hard to distinguish any difference between the 2 cases. Control starts at k=1000.

Here's a close up of a section of the uncontrolled iterations.



The SLM obviously has an effect on the objective function, but its not much, and it doesn't look like that effect more than the noise from calculating the image objective function. What is clear though is that unless I can speed things up, using the SLM at all is totally impractical. This 3 second bullshit can't go on if I'm going to really be using it.

I'm not totally convinced the SLM is doing much to the phase. I'm pretty sure its working, but since I'm not measuring the phase directly I don't know if a single wavelength of change is much.

It's clear that something is happening. Here's a plot of the target image variance (J) and slope vector norm with the SLM active. At 300 iterations it displays a focus shape (not that easy to calculate efficiently!) where the amplitude varies sinusoidally from the max displacement to min.



The good news is that the sinusoidal pattern is reflected in both objective functions. But although the change is visible, its miniscule percentage compared to what's available with the DM. As a comparison, the maximum stroke of the DM is in the range of 10 microns. The SLM is limited to 2pi rad, about 650 nm. This wouldn't matter much if I were mapping the phase measurement back to the range of [0,2pi], such as with a SRI. But since I'm not measuring the phase directly, I'm not sure how variations like that are mapped in a Hartmann sensor.

The other problem is the ludicrously slow response time of the SLR, which I still have to limit to 1/3 Hz. Given the shit this thing has given me over the last few months, I think the best plan of action is to convince my advisor to ditch the SLM and get another DM. That will require me to exhaust all obvious ways to speed things up, but at this rate that shouldn't be hard.

Damn the torpedoes

I'm finally back in the swing of things after my little excursion to the Old World. Its amazing how difficult it is to get used to "work" again.

The SLM is still heinously slow, still limited to around 1/3 Hz....yes 1 frame every 3 seconds. Personally, I think its unlikely I'll ever find a solution to this, but right now the best hope for a fix is to try using with with the other SLM's hardware (which is working fine). I'm also going to email BNL, but since I didn't write the modified code to control it using Matlab, doubt that'll result in anything useful.

Nevertheless, I'm going on as if everything was operating normally. I've devoted this week to determining if the SLM is doing anything at all. Here's a plot showing a few relevant objective functions as a focus bias was applied to the SLM with sinusoidally varying intensity. Since the bitch has been fighting me every step of the way, I was surprised to see actual sinusoidal responses here, even though their amplitude might be too small to be useful. The DM was stationary in the bias position for these experiments, and the SLM is set to zero until iteration 300:



Of course, at 1/3 Hz, these experiments take half an hour to perform. Nonetheless, there's at least an indication that the SLM may be useful to generate disturbances if I can get it moving faster. For tomorrow, I'd like begin adding such SLM disturbances into the SPGD algorithm, even though a full experiment might take a day to run (I can do it over night if necessary).

As another side project, my advisor wants me to characterize the response times of each component in my system. Unfortunately he's starting to think about real-time experiments, which seems infeasible now considering the problems I'm already having.

Europe

I'm glad that the mohawk survives in Europe. Denim capris pants? Not so much.

6.18.09

I've been fiddling more with my setup this week to get the classical and SPGD algorithms working like I expect them to, and think I about have it. Most of the hassle involved positioning the target camera. Interestingly, the SPGD algorithm was able to successfully maximize Strehl ratio from the image, but the PI controller with the WFS failed to achieve the same steady state performance. Ultimately, I think the reason for this was that the target camera was positioned slightly off plane, so an unaberrated phase didn't result in a focused spot.



Tomorrow I want to try running these algorithms with some significant, static SLM disturbance. I still can't send commands to the frickin thing faster than 1/3 Hz, so dynamic disturbance simulations are going to be sloooow until I can get that resolved.

Distracted

I got a new laptop this week, so progress was...muted. I was able to write some code to generate superpixels for the SLM, and I used it to get some preliminary poke matrix data. Basically the phase response is pretty strange as the individual pixel values change and I haven't quite figured it out yet. I did learn though that the WFS isn't capable of really detecting unit changes in pixel values, however. Luckily this doesn't really matter much since I intend to use the SLM primarily to product noise of a particular bandwidth.

I have a large trip planned at the end of next week, so I'm not sure how preoccupied I'll be over the next few days preparing. Ideally, I'd like to take another look at the experimental implementation of the gradient descent and look at how it responds to (slowly) changing disturbances. Maybe like step changes every 100 iterations or something like that.

Case of the Mondays

My adviser and one of the post-docs are out at a conference this weeks, so I'm practically alone in lab this week. Its hard to find motivation.

Today I aligned the target camera, so for the first time I have all my devices together in a single experiment: laser, DM, SLM, target camera, and WFS. Hey it only took a year!

I also finished the Simulink model that I can use to close the AO loop with any desired controller (although right now its using Simulink's integral controller block). To finish it off I added a block that reads images from the target camera and computes the image variance, although its easy to change that to any other desired performance metric.

Currently, the controller doesn't provide much improvement in the wavefront variance. I suspect this is probably because the phase is fairly flat to to begin with, and any aberration is relatively of high frequency and can't be corrected with the DM. Now that I have control of the SLM, the goal this week is to introduce (static) artificial disturbances and see how well the classical controller compensates. Next week I'd like to try some basic dynamic disturbances.

So the plan for this week:

1. Finish alignment, check tilt modes

2. Write a script to generate SLM poke matrix, begin applying static disturbances that result in a know slope disturbance.

3. Investigate performance of gradient descent algorithm using identified turbulence models.

4. Create Simulink model that implements gradient descent controller with an arbitrary performance metric.

6.4.09

I've spent most of this week porting my control algorithms to Simulink and the "suggestion" of my adviser. Conceptually this shouldn't be that hard since I already have a simulation using the experimental poke matrices. Reading the WFS image is trivial with the image acquisition toolbox. However implementing my custom code for calculating slopes and sending commands to the mirror requires me to use these Embedded Matlab blocks that translate m-files into C code for execution in Simulink, and that process involves a whole shit storm of issues. I found a pretty simple, slightly cheating, way of doing it that basically just involves calling my existing code as "extrinsic" functions that don't compile into C, but instead run in the Matlab workspace. This seems to work pretty well and right now I have my PI controller running beautifully (albeit slowly) in Simulink.

I've gotten the code necessary to control the SLM from Matlab as well, and its pretty clunky and generally crap, although I guess its functional enough to use. Getting it to work in Simulink as above though is, not surprisingly, a pain in the ass.

Tomorrow we meet with some Ophthalmologists to see if we can use any AO techniques on their retinal imaging machines. This has been done before by some people at Berkeley, so what they ask for might not be particularly novel, probably difficult (getting the optics setup), and from a control perspective not very interesting (the phase distortion of the eye is basically constant). Still, it doesn't hurt to talk.

At long last the SLM is installed and appears to be working. Here's the reconstructed phase with a test image:



See the face smiling at its own existence? Happily, this also indicates my phase reconstructor is working

Next comes controlling it from Matlab.

1. Cut a Hole in a Box

Finally received all the cables to install the SLM. In one final kick to the balls the connector for this new cable was too wide for the back of my PC case, so I had to pry this pointless plastic cover off the back. Why Dell shields their cases with useless crap like that I don't know.

I did some basic realignment and it looks like everything is attached correctly. Tomorrow I'll do some testing to see if the SLM's producing the phase changes its supposed to. After that, I need to figure out a way to get it working from Matlab.

In other news I've been working on the SPGD simulation with dynamic colored noise and multiple channels. To my surprise it actually works pretty well if the noise is kept to around 1% of the Nyquist frequency, but my adviser tells me this is pretty good. The idea is that by using the Strehl as the cost criteria potentially allows for the use of a photodiode instead of a full wavefront sensor. Thus, performance would probably only limited by the bandwidth of the DM controller, rather than the WFS framerate. Ideally, this would mean that sampling rates in the KHz range would be possible (and cheap).

Here are some typical results using 3 channels. More channels seem to work, but tuning the parameters is a bitch. In particular, the spikes in the cost function can be reduced by increasing the size of the perturbations, which (I think) get swamped by the disturbance sequence with higher noise variances.



Control begins at iteration 5000. This plot doesn't show the change in the cost function due to the dithering at each step to estimate the gradient. Its hard to tell from the plot, but the controlled cost function has an average value that is around 40% of the uncontrolled average. Note that these results don't look as good as those a couple days ago. In those results I forgot that I had a sinusoidal disturbance in addition to the colored noise sequence.

5.26.09

My gradient descent simulation is showing some progress in rejecting dynamic disturbances. Here's a plot showing the cost function (log of the Strehl ratio using the Marichal approx), noise term, and control sequence using a noise bandwidth that's 1% of the Nyquist frequency. Control starts at 5000 iterations.




My view at the moment is that the command sequence essentially acts as a first order system, exponentially changing in response to a step disturbance. The transient response of this system is governed by the size of the optimization step. So as long as the period of the dynamic disturbances are sufficiently larger than the rise time, the algorithm shouldn't have a problem rejecting it.

This analogy isn't exactly correct since the gradient must be estimated at each step, and the step size is only constant if the normalize gradient and constant gain are used, which isn't always the case. Still, the problem basically boils down to finding a control gain large enough to provide convergence, but small enough to be stable. Even more, since the objective function is basically quadratic, there's probably some way to quantify the optimal step size, but I'm too lazy at the moment to spend much time on that street.

The current simulation is very simple, using a single channel with no poke matrices or actuator nonlinearities. The next step still be to gradually introduce these features until I have a realistic model. If this works at all, it might be worth looking into other first-order algorithms that have even faster convergence properties. i.e. pseudo-Newton and Nesterov's.