debugging, adding features

Author: Ali Basden

README_mac

@@ -0,0 +1,122 @@
+Instructions for installing DASP (the Durham AO Simulation Platform) onto Mac OS X using MacPorts
+
+
+Date: 2013/11/05 (NAB), 2013/12/09 (NAB,ZG,DH)
+
+Tested: 10.5.8 and 10.7.5 and 10.9.??
+
+Authors: Angela Cortés, Nazim Bharmal, and Alastair Basden
+
+
+There are three options for using DASP on a computer that runs the Mac OS X OS.
+
+
+    The easiest approach is to run DASP from a virtual machine using VirtualBox---see NAB for a copy of the VDI.
+    It is possible (and has been done) to install all packages by hand into an existing Python installation. In some ways this can be better, but it took ~12 hours and required editing packages to ensure correct installation.
+    Install all the pre-requisites that DASP requires for installation using MacPorts as a package manager. Note that this will install a version of DASP that isn't MPI-compatible as well as potentially another copy of Python and all its associated packages.
+
+All commands to be entered at the command line are indented. Their output is also indented but only if it is important. Below are the commands for using MacPorts. First install that piece of software and ensure it can compile. Then update the packages and install Python, Numpy, and Scipy:
+
+
+sudo port selfupdate
+
+  sudo port install py27-numpy py27-scipy
+
+
+Check that the Python interpreter,
+
+
+python -c 'import sys ; print(sys.executable)'
+
+/opt/local/bin/python
+
+
+is installed in a different directory from the system Python interpreter. This does mean you are installing another copy of Python onto your computer but the package dependencies are handled by MacPorts.
+
+
+If the line above doesn't begin with '/opt/local' then it is probable you have called the system Python interpreter. Try using 'python2.7' instead for all commands below. This would also imply changing the Makefiles in DASP.
+
+
+Now continue by installing ATLAS (optional)
+
+
+sudo port install atlas
+
+
+which if it fails is OK. Then install FFTW,
+
+
+sudo port install fftw-3-long fftw-3-single
+
+
+and then PyScientific and GNU Scientific Library plus the Python bindings,
+
+
+        sudo port install py-scientific py27-gsl
+
+
+Finally we add GTK2, and the Python bindings together with the correct back-end for Matplotlib,
+
+        sudo port install gtk2 py-pygtk
+
+        sudo port install py27-matplotlib +gtk2
+
+
+Note that most installs were using the 'py' prefix but for GSL and matplotlib, it was necessary to use 'py27'.
+
+(The software installed can also be summarized as: Python, ATLAS (LAPACK and BLAS), complete FFTW3, GTK and PyGTK, scipy, Scientific Python, GNU Scientific Library and the Python binding, matplotlib with the GTK backend.)
+
+
+mkdir DASP
+
+cd DASP
+
+git clone gitaosim@elise.cfai.dur.ac.uk:/home/gitaosim/aosim.git
+
+
+Note to access the GIT repository on elise, you will have to ask Alastair Basden for the password to the gitaosim user account.
+
+
+cd aosim
+
+
+Now you will attempt a compilation which will succeed iff the include directories in the makefiles include,
+
+
+/opt/local/include
+
+
+so if the following command fails,
+
+
+make -s
+
+
+then I would suggest checking the include directories first. A key file that could need adjusting is,
+
+
+aosim/cmod/src/Makefile.Darwin
+
+
+Once compilation is complete, follow the final instruction which will be similar to
+
+
+Remember to export PYTHONPATH=$PYTHONPATH:/Users/nab/sauce/aosim/working and add gui/bin to your path (export PATH=$PATH:/Users/nab/sauce/aosim/working/gui/bin ).
+
+
+where '/Users/nab/sauce/aosim/working' is replaced by the directory where aosim is installed. To test correct installation, type
+
+
+simctrl.py &
+
+
+to bring up the DASP control GUI. For further instructions and simulation examples, contact anyone using DASP. Obvious contacts are,
+
+
+Ali Bharmal	for Mac OS X/Darwin questions
+
+n.a.bharmal@dur.ac.uk
+
+Alastair Basden	for DASP questions
+
+a.g.basden@dur.ac.uk

base/readConfig.py

@@ -349,6 +349,7 @@ def end_element(self,name):
                     print sys.exc_value
                     v=None
                     self.error=1
+                    raise
                 if hasattr(self.this,self.curmodule):
                     if hasattr(getattr(self.this,self.curmodule),attrs["name"]):
                         if bInfo==1:#overwrite prev value
@@ -510,7 +511,7 @@ def setSearchOrder(self,searchOrder):
         @type searchOrder: List
         """
         self.searchOrder=searchOrder
-    def getVal(self,varname,default=None,searchOrder=None,raiseerror=1):
+    def getVal(self,varname,default=None,searchOrder=None,raiseerror=1,warn=1):
         """Return the value of a variable stored in the XML file.
         This would be used e.g. myvar=x.getVal("myvar") where x is the
         instance of AOXml (self).  A default value can be given.  If no default
@@ -549,7 +550,8 @@ def getVal(self,varname,default=None,searchOrder=None,raiseerror=1):
                 print "ERROR: value not found %s"%str(varname)
                 raise Exception("ERROR: value not found: %s %s"%(str(varname),str(searchOrder)))
             else:
-                print "WARNING: using default value of %s for %s %s"%(str(val),str(varname),str(searchOrder))
+                if warn:
+                    print "WARNING: using default value of %s for %s %s"%(str(val),str(varname),str(searchOrder))
         return val
 
     def setVal(self,varname,value,searchOrder=None,raiseerror=1):

base/saveOutput.py

@@ -166,7 +166,7 @@ def generateNext(self,msg=None):
                     newdim=int(key[10:])+1
                     util.FITS.WriteKey(self.ff,"NAXIS%d"%self.nd,str(newdim))
                 else:
-                    print "saveOutput: waiting for data but not valid (debug=%s)"%debug
+                    print "saveOutput: waiting for data but not valid (debug=%s)"%self.debug
                     self.dataValid=0
         else:
             self.dataValid=0

base/splitOutput.py

@@ -2,7 +2,7 @@
 import types
 import base.fwdMsg
 #import Numeric
-#import numpy
+import numpy
 import cmod.utils
 import base.aobase
 class splitOutput(base.aobase.aobase):
@@ -43,7 +43,7 @@ def __init__(self,parent,config,args={},forGUISetup=0,debug=None,idstr=None):
         #Code must be python code that extracts part of an array from parentData, and writes it to outputData, e.g. "outputData=parentData[0:10]".  This can be as complex as you like.
         self.outputData=None
         self.compiledCode=compile(self.code,"<string>","exec")
-        self.shape=None
+        self.shape=config.getVal("splitOutputShape",raiseerror=0)
         self.dtype=None
         self.feedback=0#not a feedback object by default...
         self.forGUISetup=forGUISetup
@@ -86,7 +86,7 @@ def finalInitialisation(self):
         if self.initialised==0:
             dtype=None
             if self.shape=="fromparent":#attempt to get shape from parent (could be dangerous if parent not yet created properly).
-                d={"parentData":self.parent.outputData,"outputData":self.outputData,"parent":self.parent}
+                d={"parentData":self.parent.outputData,"outputData":self.outputData,"parent":self.parent,"numpy":numpy,"config":self.config}
                 exec self.compiledCode in d
                 self.shape=d["outputData"].shape
                 print "splitOutput got shape of %s from parent (code=%s)"%(self.shape,self.code)
@@ -138,7 +138,7 @@ def generateNext(self,msg=None):
             if self.newDataWaiting:
                 if self.parent.dataValid==1:
                     self.dataValid=1
-                    d={"parentData":self.parent.outputData,"outputData":self.outputData,"parent":self.parent}
+                    d={"parentData":self.parent.outputData,"outputData":self.outputData,"parent":self.parent,"config":self.config,"numpy":numpy}
                     exec self.compiledCode in d
                     outputData=d["outputData"]
                     if outputData.shape!=self.shape:
@@ -160,7 +160,7 @@ def generateNext(self,msg=None):
 
 ##                     self.outputData=cmod.utils.arrayFromArray(self.parent.outputData[self.startOffset:self.endOffset],self.shape,self.dtype)
                 else:
-                    print "splitOutput: waiting for data but not valid (debug=%s)"%debug
+                    print "splitOutput: waiting for data but not valid (debug=%s)"%self.debug
                     self.dataValid=0
         else:
             self.dataValid=0

util/Ctrl.py

@@ -167,7 +167,7 @@ def __init__(self,globals=None,paramfile=[],debug=None):
         self.config.this.simID=self.simID
         self.sockConn=util.SockConn.SockConn(self.config.getVal("testrunport",default=9000)+self.rank,globals=self.globals,startThread=1,listenSTDIN=self.listenSTDIN)
         os.nice(self.config.getVal("nice",default=nice))
-        if self.config.getVal("connectPortDict",default=1):
+        if self.config.getVal("connectPortDict",default=0,warn=0):
             try:
                 t=threading.Thread(target=self.connectPortDict)
                 t.daemon=True

util/FITS.py

@@ -227,7 +227,7 @@ def ReadHeader(filename, asFloat = 1) :
 	buffer = file.read(2880)
     return( { 'raw' : rawHeader, 'parsed' : header} )
 
-def MakeHeader(shape,dtype,extraHeader=None,doByteSwap=1,extension=0):
+def MakeHeader(shape,dtype,extraHeader=None,doByteSwap=1,extension=0,splitExtraHeader=0):
     """Return a text string which can be used as a header"""
     if dtype=="b": bitpix=8
     elif dtype=="B": bitpix=8
@@ -265,6 +265,10 @@ def MakeHeader(shape,dtype,extraHeader=None,doByteSwap=1,extension=0):
 		   key != 'BITPIX' and \
 		   key[:5] != 'NAXIS' and \
 		   key != 'END' :
+                    if splitExtraHeader:
+                        while len(rec)>80:
+                            header.append(rec[:80])
+                            rec=string.ljust(key,8)+"= "+rec[80:]
 		    header.append(rec)
     header.append('END')
     header = map(lambda x: string.ljust(x,80)[:80], header)

util/computeRecon.py

@@ -7,6 +7,7 @@
 """
 import util.FITS,numpy
 import sys,time,os
+import util.gradientOperator
 #import util.dot as quick
 #sys.path.insert(0,"/home/ali/c/lapack")
 #import svd
@@ -30,6 +31,24 @@
 
 import cmod.svd,scipy.sparse
 
+def keyboard(banner=None):
+    """A class that allows us to break to python console on exception."""
+    import code, sys
+
+    # use exception trick to pick up the current frame
+    try:
+        raise None
+    except:
+        frame = sys.exc_info()[2].tb_frame.f_back
+
+    # evaluate commands in current namespace
+    namespace = frame.f_globals.copy()
+    namespace.update(frame.f_locals)
+
+    code.interact(banner=banner, local=namespace)
+
+
+
 class sparseHolder:
     def __init__(self):
         self.data=None
@@ -54,6 +73,12 @@ def __init__(self,pmxfname,rcond=0.1,regularisation=0.):
         self.timename=pmxfname[:-5]+"_timing.txt"
         self.cntname=pmxfname[:-5]+"_cnt.txt"
         self.regularisation=regularisation
+
+    def initFromParams(paramList=["params.xml"],batchno=0):
+        import base.readConfig
+        self.c=base.readConfig.AOXml(paramList,batchno=batchno)
+        c=self.c
+
     def log(self,txt):
         f=open(self.timename,"a")
         f.write(txt)

util/dm.py

@@ -8,6 +8,25 @@
 import util.dot as quick
 import time
 
+
+def calcNactList(config,batchno,reconidstr):
+    if type(config) in [type([]),type("")]:
+        import base.readConfig
+        config=base.readConfig.AOXml(config,batchno=batchno)
+    config.setSearchOrder(["tomoRecon_%s"%reconidstr,"tomoRecon","globals"])
+    dmObj=config.getVal("dmObj")
+    pupil=config.getVal("pupil")
+    atmosGeom=config.getVal("atmosGeom")
+    dmList=dmObj.makeDMList(reconidstr)
+    nacts=0
+    nactsList=[]
+    for dm in dmList:
+        if dm.zonalDM:
+            tmp=dm.computeDMPupil(atmosGeom,centObscuration=pupil.r2,retPupil=0)
+            nactsList.append(int(tmp[0].sum()))
+        else:#modal DM
+            nactsList.append(dm.nact)
+    return nactsList
 """Contains:
 dmInfo class - info about a given DM
 dmOverview class - overview of all DMs in a system

util/sci.py

@@ -992,16 +992,18 @@ def doScienceCalc(self,inputData,control,curtime=0):
                         if self.luckyCnt==0:
                             if self.luckyImg==None:
                                 self.luckyImg=numpy.empty((self.nimg,self.nimg),self.fpDataType)
-
+                            self.luckyRms=self.phaseRMS
                             self.luckyImg[:]=self.instImg
                         else:
+                            self.luckyRms+=self.phaseRMS
                             self.luckyImg+=self.instImg
                         self.luckyCnt+=1
                         if self.luckyCnt>=self.luckyNSampFrames:
                             self.luckyCnt=0
                             #Now do the lucky calculations.
                             self.luckyImg/=self.luckyImg.sum()#normalise it
                             self.computeScientificParameters(self.luckyImg,self.luckyDict)
+                            self.luckyDict["rms"]=self.luckyRms/self.luckyNSampFrames#the mean RMS phase that went into this image.
                             if self.luckyHistory==None:
                                 self.luckyHistoryKeys=self.luckyDict.keys()
                                 self.luckyHistory=numpy.zeros((len(self.luckyHistoryKeys),self.luckyHistorySize),numpy.float32)