python 3 converstion of poretools

.gitignore

@@ -2,3 +2,6 @@
 build/
 dist/
 *.egg-info
+ENV*
+.coverage
+.gitignore

.travis.yml

@@ -0,0 +1,16 @@
+# travis ymal file for python3 development
+
+language: python
+
+python:
+  - 2.7
+  - 3.5
+  - 3.6 
+
+install:
+  - pip install -r requirements.txt
+  - python setup.py install
+script:
+  - python setup.py install
+  - cd test_data/
+  - bash poretools-tests.sh

README.md

@@ -1,3 +1,5 @@
+[![Build Status](https://travis-ci.org/skbrimer/poretools.svg?branch=master)](https://travis-ci.org/skbrimer/poretools)  ![python version](https://img.shields.io/badge/python-2.7%2C%203.3%2C%203.4%2C%203.5%2C%203.6-blue.svg)
+
 ### poretools: a toolkit for working with nanopore sequencing data from Oxford Nanopore.
 
 *Nick Loman and Aaron Quinlan*

TODO.md

@@ -0,0 +1,97 @@
+# TO-DO For python3 and python2 compatiablity of poretools
+
+## Just conversion to py3, py2 not tested... yet 
+
+1. **stats** - printing throughout the loop for each addition of reads, failing tests. Expected outcome is one output. 
+ -1. also not generating N50 or N75, don't know if this is a byproduct of the print issue or a sepperate issue
+
+2. **qualdist** - seems to be working, but has too many decimals and fails test. 
+
+3. **nucdist** - seems to be working, but has too many decimals and fails test. 
+
+4. ~~**yield_plot** 
+Traceback (most recent call last):
+  File "/home/sbrimer/Documents/poretools/ENV/bin/poretools", line 11, in <module>
+    load_entry_point('poretools==0.6.0', 'console_scripts', 'poretools')()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 539, in main
+    args.func(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 55, in run_subtool
+    submodule.run(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/yield_plot.py", line 96, in run
+    plot_collectors_curve(args, start_times, read_lengths)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/yield_plot.py", line 35, in plot_collectors_curve
+    d = {'start': [start_times[n] for n in xrange(0, len(start_times), step)],
+NameError: name 'xrange' is not defined~~
+
+5. **squiggle** - need to rework next ~~and import iterateitems~~
+`WARNING:poretools:Could not extract template events for read: test_data/2016_3_4_3507_1_ch13_read1474_strand.fast5.
+`
+
+6. ~~**time**
+Traceback (most recent call last):
+  File "/home/sbrimer/Documents/poretools/ENV/bin/poretools", line 11, in <module>
+    load_entry_point('poretools==0.6.0', 'console_scripts', 'poretools')()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 539, in main
+    args.func(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 55, in run_subtool
+    submodule.run(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/times.py", line 25, in run
+    start_time = fast5.get_start_time()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/Fast5File.py", line 587, in get_start_time
+    exp_start_time  = self.get_exp_start_time()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/Fast5File.py", line 431, in get_exp_start_time
+    if self.keyinfo['tracking_id'].attrs['exp_start_time'].endswith('Z'):
+TypeError: endswith first arg must be bytes or a tuple of bytes, not str~~
+
+7. ~~**occupancy** Traceback (most recent call last):
+  File "/home/sbrimer/Documents/poretools/ENV/bin/poretools", line 11, in <module>
+    load_entry_point('poretools==0.6.0', 'console_scripts', 'poretools')()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 539, in main
+    args.func(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 55, in run_subtool
+    submodule.run(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/occupancy.py", line 81, in run
+    plot_performance(parser, args, tot_reads_per_pore)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/occupancy.py", line 39, in plot_performance
+    d = {'rownum': range(1,17)*32, TypeError: unsupported operand type(s) for *: 'range' and 'int~~
+8. ~~**combine** - not sure about this one, may need work or better understanding of how to use it.~~ User error :/
+
+9. ~~**index**
+Traceback (most recent call last):
+  File "/home/sbrimer/Documents/poretools/ENV/bin/poretools", line 11, in <module>
+    load_entry_point('poretools==0.6.0', 'console_scripts', 'poretools')()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 539, in main
+    args.func(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/poretools_main.py", line 55, in run_subtool
+    submodule.run(parser, args)
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/index.py", line 35, in run
+    start_time = fast5.get_start_time()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/Fast5File.py", line 587, in get_start_time
+    exp_start_time  = self.get_exp_start_time()
+  File "/home/sbrimer/Documents/poretools/ENV/lib/python3.5/site-packages/poretools-0.6.0-py3.5.egg/poretools/Fast5File.py", line 431, in get_exp_start_time
+    if self.keyinfo['tracking_id'].attrs['exp_start_time'].endswith('Z'):
+TypeError: endswith first arg must be bytes or a tuple of bytes, not str~~
+
+10. **readstats**
+~~start_time channel_number  read_number template_events complement_events
+poretools internal error in file 'test_data/YYYYMMDD_HHMM_SampleID/Fail/1/2016_3_4_3507_1_ch120_read443_strand.fast5': unknown HDF5 structure: can't find read block item
+                 Please report this error (with the offending file) to:
+                 https://github.com/arq5x/poretools/issues~~  ** Not and error, test data has older file structure **
+
+11. ~~ **logger** - doesn't log currently?~~ prints to standard out
+
+
+## Working Modules - for py3
+1. **fastq**
+2. **fasta**
+3. **winner**
+4. **qualpos**
+5. **hist**
+6. **tabular**
+7. **metadata**
+8. **combine** 
+9. **index**
+10. **times**
+11. **occupancy**
+12. **yield_plot**
+13. **readstats**

appveyor.yml

@@ -91,22 +91,22 @@ install:
   # compiled extensions and are not provided as pre-built wheel packages,
   # pip will build them from source using the MSVC compiler matching the
   # target Python version and architecture
-  #
+
   - ps: if ($env:PYTHON_ARCH -eq "32") { `
-        Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/h5py-2.5.0-cp27-none-win32.whl" -OutFile "C:\HDF5\h5py-2.5.0-cp27-none-win32.whl";
-        pip install C:\HDF5\h5py-2.5.0-cp27-none-win32.whl `
-        } else { `
-        Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/h5py-2.5.0-cp27-none-win_amd64.whl" -OutFile "C:\HDF5\h5py-2.5.0-cp27-none-win_amd64.whl";
-        pip install C:\HDF5\h5py-2.5.0-cp27-none-win_amd64.whl `
+        Invoke-WebRequest "https://pypi.python.org/packages/4e/5b/1da1bf9537224adb6ca9148ca34fa9726a878660bbb45705747402d5a854/h5py-2.7.1-cp27-cp27m-win32.whl#md5=849ff329ce2cebb4d2c858eac0d2ff96" -OutFile "C:\HDF5\h5py-2.5.0-cp27-cp27m-win32.whl";
+        pip install C:\HDF5\h5py-2.5.0-cp27-cp27m-win32.whl `
+       } else { `
+        Invoke-WebRequest "https://pypi.python.org/packages/98/43/cb0345e3e47cbc1958200d2ecb536a202550fc58c54134aab6b0bb1d9794/h5py-2.7.1-cp27-cp27m-win_amd64.whl#md5=02f06d0f8208832c4b5cdddcad954932" -OutFile "C:\HDF5\h5py-2.5.0-cp27-cp27m-win_amd64.whl";
+        pip install C:\HDF5\h5py-2.5.0-cp27-cp27m-win_amd64.whl `
         }
 
-  - ps: if ($env:PYTHON_ARCH -eq "32") { `
-        Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/matplotlib-1.5.1-cp27-none-win32.whl" -OutFile "C:\HDF5\matplotlib-1.5.1-cp27-none-win32.whl";
-        pip install C:\HDF5\matplotlib-1.5.1-cp27-none-win32.whl `
-        } else { `
-        Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/matplotlib-1.5.1-cp27-none-win_amd64.whl" -OutFile "C:\HDF5\matplotlib-1.5.1-cp27-none-win_amd64.whl";
-        pip install C:\HDF5\matplotlib-1.5.1-cp27-none-win_amd64.whl `
-        }
+  #- ps: if ($env:PYTHON_ARCH -eq "32") { `
+  #      Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/matplotlib-1.5.1-cp27-none-win32.whl" -OutFile "C:\HDF5\matplotlib-1.5.1-cp27-none-win32.whl";
+  #      pip install C:\HDF5\matplotlib-1.5.1-cp27-none-win32.whl `
+  #      } else { `
+  #      Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/matplotlib-1.5.1-cp27-none-win_amd64.whl" -OutFile "C:\HDF5\matplotlib-1.5.1-cp27-none-win_amd64.whl";
+  #      pip install C:\HDF5\matplotlib-1.5.1-cp27-none-win_amd64.whl `
+  #      }
 
   #- ps: Invoke-WebRequest "https://pypi.python.org/packages/2.7/h/h5py/h5py-2.5.0.win32-py2.7.exe" -OutFile "C:\HDF5\h5py-2.5.0.win32-py2.7.exe"
   #- ps: Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/h5py-2.5.0-cp27-none-win%PYTHON_ARCH%.whl" -OutFile "C:\HDF5\h5py-2.5.0-cp27-none-win%PYTHON_ARCH%.whl"
@@ -119,12 +119,27 @@ install:
     #- "%CMD_IN_ENV% pip install C:\\HDF5\\matplotlib-1.5.1-cp27-none-win32.whl"
 
 
-  - ps: Invoke-WebRequest "http://www.lfd.uci.edu/~gohlke/pythonlibs/djcobkfp/seaborn-0.7.0-py2.py3-none-any.whl" -OutFile "C:\HDF5\seaborn-0.7.0-py2.py3-none-any.whl"
+  #- ps: wget "https://www.lfd.uci.edu/~gohlke/pythonlibs/seaborn-0.8.1-py2.py3-none-any.whl" -OutFile "C:\HDF5\seaborn-0.8.1-py2.py3-none-any.whl"
     #- "%CMD_IN_ENV% C:\\HDF5\\h5py-2.5.0.win32-py2.7.exe"
-  - ps: pip install C:\\HDF5\\seaborn-0.7.0-py2.py3-none-any.whl
-
+  #- ps: pip install C:\\HDF5\\seaborn-0.8.1-py2.py3-none-any.whl
+  - '"C:\Program Files (X86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat"'
+  # Remove the VS Xamarin targets to reduce AppVeyor specific noise in build
+  # logs.  See also http://help.appveyor.com/discussions/problems/4569
+  - del "C:\Program Files (x86)\MSBuild\12.0\Microsoft.Common.targets\ImportAfter\Xamarin.Common.targets"
+  - ps: Start-FileDownload https://github.com/glfw/glfw/archive/3.2.1.zip -FileName glfw-3.2.1.zip
+  - unzip glfw-3.2.1.zip
+  - cd glfw-3.2.1
+  - cmake -G "Visual Studio 12" -DBUILD_SHARED_LIBS=ON -DGLFW_BUILD_EXAMPLES=OFF -DGLFW_BUILD_TESTS=OFF -DGLFW_BUILD_DOCS=OFF .
+  - cmake --build . --config Release --target install -- /logger:"C:\Program Files\AppVeyor\BuildAgent\Appveyor.MSBuildLogger.dll" /verbosity:minimal
+  - cd ..
+
+before_build:
+  - if not exist build\NUL mkdir build
+  - cd build
 
 build_script:
+  - cmake -G "Visual Studio 12" -Dfreetype-gl_BUILD_APIDOC=OFF -Dfreetype-gl_BUILD_TESTS=OFF "-Dglfw3_DIR=C:/Program Files (x86)/GLFW/lib/cmake/glfw3" ..
+  - cmake --build . --config RelWithDebInfo -- /logger:"C:\Program Files\AppVeyor\BuildAgent\Appveyor.MSBuildLogger.dll" /verbosity:minimal
   # Build the compiled extension
   - "%CMD_IN_ENV% python setup.py build"
   - "%CMD_IN_ENV% python setup.py install"