Real and Imaginary parts of Fourier transforms compared to Matlab/python

I am trying to write a program in Igor that recreates one that I have in both Matlab and python.

The program should take a Fourier transform of a 2D image (the data is actually a hologram). When I compare the magnitude images with those from Matlab/python, they agree well. However, when I look at the real/imaginary parts the are completely scrambled. The values oscillate from very negative to very positive in the space of one pixel. I am guessing that this must be something to do with the phase fluctuating between pi and -pi, however, I could't seem to figure out a way to stop it.

Is there something stupid that I am doing? With matlab/python the real and imaginary parts look fine without having to do anything tricky.
You haven't given many details about how you are doing the FFT (there are a lot of flag options in Igor Pro!). Even so, the fact that your magnitude images agree argues for your using the correct scaling and centering choices. A likely culprit that remains is the choice of sign for sqrt(-1) ['i' if you are physicist, or 'j' if you are an electrical engineer]. Not all FFT programs use common signs. Try taking the complex conjugate of your complex Igor output ( conj(..) ).
I tried the conjugate, command but it had no significant effect. I have tried lots of things forward transformation backwards transformation. Making the initial image complex, but with the imaginary part zero, multiplying the whole image by a complex number etc. However all result in the real and imaginary parts rapidly switching sign on a per pixel basis so that the magnitude remains smooth.
In addition to the comments above there are other possible explanations:
1. your transform on that "other program" may not use symmetry or does not center the results.
2. depending on the program and the dimensions of the image there may be padding involved.
3. If your input is a square array try a rectangular array (different dimension) to make sure that you are not switching between row-major/column major.
I agree with AG's comment 1, and now think it the more likely explanation. My earlier statement about the correctness of your centering was probably wrong.

A very useful Example file for you to look at is "FFT Swapping Demo.pxp", found in the Examples directory or via the main File->Example Experiments->analysis route.

The consequence of a translation (co-ordinate offset) in direct space is a large linearly varying phase shift in Fourier space, which sounds like what you are reporting.