Open writing projects/Scientific Programming with Python and Subversion/Outline

Outline

• sections marked with '(modular)' can be re-written using a different technology (i.e. git instead of svn)

0 Introductory remarks

• Why this book
  • motivation - lots of training in what science to do with computers, but little training in how to do it
  • for beginners - assumes no prior knowledge, introduces tools as they are needed in a typical scientific investigation using computers
  • for experienced scientists - introducing new tools to do some of these tasks
  • goal - to make managing projects easier, but more importantly to promote good scientific practice through these methods
• Introduce scientific themes throughout the book
  • Some bioinformatics theme - maybe use an example from one of the NCBI coffee breaks
  • Some physics theme?

1 Source Control Management with Subversion

• What is source control?
  • like Word 'track changes' or wiki 'history' but for all the files in a project.
  • A way to keep a history of every step in a process.
  • Not only for computer code, but for data, plots, paper manuscripts, etc.
• Introduction to subversion (modular)
  • What is a repository
  • How to create a repository
  • How to make bosic commits
  • Seeing differences between versions
  • Retrieving past versions
  • Collaboration using subversion
• Advanced Topics
  • Branching and Merging

2 An Introduction to Python

• What is python
  • computer language that offers easy access to high-level functions, and has a large and growing community of scientific users
• Why python
  • python code looks clean - easy to understand your code a week later, or collaborators code
  • everything can be done in python from data generation to analysis to plots making every aspect of the project consintent
  • these together promote good scientific practices (data integrity, data reproduceability)
• An introduction to python (modular)
  • variable assignment
  • basic control structures
  • functions
  • package structure and import
  • objects (just like packages)

3 Making scientific plots with python

• An introduction to matplotlib (modular)
  • basic functionality - simple line, bar, histogram plots
  • more sophisticated graphics - insets, labeling with text, drawing arrows
  • interactive graphics - adjusting parameters for real-time fitting
• An example project use of matplotlib
  • bioinformatics
  • physics

4 Crunching numbers with python

• Python community modules (modular)
  • using numpy for matrix manipulations
  • using the scipy project tools
  • interacting with the Gnu Scientific Library
• An example project
  • bioinformatics
  • physics

5 Unit testing for scientists

• What is unit testing?
  • A way to generate automated tests of small units of code
• Why do unit testing?
  • example: switching a sorting algorithm - how do you know the code works the same way
    • typically done by 'eye' by running the code manually and looking at output
    • with unit tests can see if the code failed, and if it did, where exactly
• Using python and nose to write unit tests?
  • example of test code, and how to run the tests
    • bioinformatics
    • physics
• How do I know which tests to write?
  • (This one is hard)

6 A Complete case study

• this section could be omitted initially - ideally we could have an svn repo set up for people to pull from to look at the code examples at each step of the way
• go through from start to finish
  • initially create a repository
  • the first code
  • the first tests
  • moving on

7 Advanced topic - using SWIG and psyco to speed up python code

• this section could be omitted initially
• What if python is not fast enough for my project?
  • Several options:
    • Use psyco to 'compile' the python code
    • Identify the slow parts and write them in C/C++ and bind them to python using SWIG
• Using psyco
• Using C with SWIG