Skip to content

New Release

Jump to bottom Edit New page
Aaron Meurer edited this page Nov 21, 2012 · 89 revisions

TODO Automate this whole process. See https://groups.google.com/d/topic/sympy/UfNhyFv-oMg/discussion.

This is a step by step guide on how to make a new release of SymPy. If you are following it to make a new release, please update it with any corrections or problems that you come across. This guide was written with 0.7.0 as the sample release. Replace "0.7.0" in the commands with the version that you are releasing, and "0.6.7" with the previous version.

How to Make a New Release

Proposed workflow (for the 0.7.0 release, for example)

  1. Fix all blockers (i.e., issues marked with the label Milestone-Release0.7.0), or postpone them if they aren't too important and will take too much work to fix. While you're doing this, it's a good idea to start writing the release notes. See "Release notes" below.
  2. Create a new branch 0.7.0, in the sympy repo (the official one) and do the release patches in it.
  3. Create a pull request for the sympy branch to master. Do not merge this until after you have released. Note that GitHub won't let you create a pull request until made at least one commit on the branch. People can then run sympy-bot on the branch with sympy-bot review <PR number> -m HEAD.
  4. Continue in master just like if nothing was happening. Go ahead and change the version number in master (e.g., to 0.7.0-git). Note that because of this, you'll have to do the final merge manually.
  5. Create rc1 from the 0.7.0 branch, test it, push fixes to 0.7.0. do rc2 etc. do final release.
  6. Wait for some time and see if all is ok, possibly do some release fixes (like wait for one week).
  7. Merge 0.7.0 with master, delete the branch.
  8. If more fixes are needed, simply fork from the latest 0.7.0 tag, and push more fixes, release and merge.

Note that some of the stuff below tests things in master (like the test_pure* scripts). If you choose this route, you may need to modify them to test the 0.7.0 branch).

Are All Tests Included?

As directed in setup.py, check the tests list using the command:

$ python bin/generate_test_list.py

Are All Modules Included?

As directed in setup.py, check the module list using the command:

for i in `find sympy -name __init__.py | rev | cut -f 2- -d '/' | rev | egrep -v "^sympy$" | egrep -v "tests$" `; do echo "'${i//\//.}',"; done | sort

Does It Depend on Python 2.5-2.7, 3.2-3.3 only?

Run the tests sandboxed using tox. See Using-Tox and the tox.ini.sample file in the git repo.

Speed of Import

After several runs, this should be around:

$ ipython

In [1]: time import sympy
CPU times: user 0.08 s, sys: 0.00 s, total: 0.09 s
Wall time: 0.09

You can also use the test_import utility like this:

$ bin/test_import
Note: the first run (warm up) was not included in the average + std dev
All runs (including warm up):
[0.112630844116, 0.112730026245, 0.13080096244799999, 0.11622381210299999, 0.113546848297, 0.13612699508699999, 0.115744113922, 0.112670898438, 0.11331987381, 0.13219594955399999, 0.113155126572]
Number of tests: 10
The speed of "import sympy" is: 0.119651 +- 0.008924

Run this in the last release and see it the tolerance intervals overlap. If they do, there is no statistically significant change in the import time. If they do not, and the new time is slower, you should investigate.

You should not compare them with the timings pasted here, as they were probably run on a different machine, and are thus not comparable to yours.

Do All Tests Run in the Isolated Environment?

Do this on the git version:

$ bin/test_isolated
Generating py.test isolated testsuite...
Done. Run '/tmp/test_sympy.sh'.
$ /tmp/test_sympy.sh |less

And check manually, that no py.test run generated a stack trace. (search for "COMMIT" in the less environment, when all tests have finished)

Do the Slow Tests Pass?

Run the slow tests with

$ ./bin/test --slow

Note that these will take a really long time to finish. This should be tested in all relevant configurations.

Do all Examples Work?

$ examples/all.py -w

Make sure no traceback is generated and this is printed at the end:

NO FAILED EXAMPLES

Note that the -w will make it also run windowed examples, which will require things like pyglet and matplotlib to be installed to work.

Also, make sure that all the examples are included in all.py by comparing the included modules with the output of ls -R examples.

You also should check that all the notebooks are OK. For now, this means opening all the notebooks in the latest version of IPython and making sure that everything renders OK and that if you re-execute all the cells, that nothing changes. Also, make sure that the cell numbering is 1, 2, 3... (you can get this by restarting the IPython kernel and re-executing all cells. In the future, we will hopefully automate this process.

Does pyglet Work?

Create a file plotting.py with

from sympy import Symbol, cos, sin, Plot, log, tan
from sympy.abc import x, y
print Plot(cos(x)*sin(y), sin(x)*sin(y), cos(y)+log(tan(y/2))+0.2*x, [x, -0.00,
    12.4, 40], [y, 0.1, 2, 40])

Then I create a file tox.ini.plotting with

[tox]
envlist = py25,py26,py27

[testenv:py25]
basepython = /Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5
deps=ctypes
    http://pyglet.googlecode.com/files/pyglet-1.1.4.tar.gz
commands = pythonw plotting.py

[testenv:py26]
deps=http://pyglet.googlecode.com/files/pyglet-1.1.4.tar.gz
basepython = /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
commands = pythonw plotting.py

[testenv:py27]
deps=http://pyglet.googlecode.com/files/pyglet-1.1.4.tar.gz
basepython = /Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
commands = pythonw plotting.py

(Python 2.5 requires ctypes, see http://docs.sympy.org/dev/modules/plotting.html). If you do not use Mac OS X, you will need to modify the basepython lines, or just remove them.

Then, run

$tox -c tox.ini.plotting

And verify that the plots open correctly all three times.

Note, because of the pythonw calls, the above tox call does not actually run from the virtualenv. Therefore, you will have to have Pyglet installed in each version of Python. You can get it at http://www.pyglet.org/. We should support whatever the latest version is.

In Linux, it might work without the pythonw. So if it says it can't find pythonw, try changing it to just python.

Check All Tests in sympy/external

Currently numpy, scipy, sage, and some fortran compilers. Open the test files to see what libraries you need to install for them to run. To run the sage tests, you have to run sage -python bin/test sympy/external/tests/test_sage.py (see the docstring of the sympy/external/tests/test_sage.py file).

Check Docs

$ cd doc
$ make clean
$ make html
$ make htmli18n
$ # open _build/html/index.html in your preferred web browser

Note that Sphinx shouldn't output any errors when building the docs (many errors result in parts of the documentation not being included, so this is important).

Also check that the pdf docs work. This require LaTeX, XeTeX, DejaVu Sans Mono, and Inkscape.

$ make clean
$ make latex
$ cd _build/latex
$ make all

There will be a bunch of warnings in the LaTeX, but if it finishes building, it will be OK. Open the built pdf (it will be called sympy-0.7.0-git.pdf, or something like that), and make sure it looks OK.

Finally, do make clean. You will want to rebuild the docs again after you change the version number, so that it is correct in the docs (or just do that part first :)

Note that due to quirkinesses of Sphinx, you will want to run make clean between each doc build, just to be sure that everything come out right.

Update versions and authors

Update the version in src/conf.py, update the list of contributors both in docs and in AUTHORS, using (for example, for the 0.7.0 release):

$ git shortlog -ns sympy-0.6.7..

(use the tag of the latest release) and then update the information what each contributor changed using

$ git shortlog -n sympy-0.6.7..

See also the bit about .mailmap below for another useful git command to use here.

Does the Tarball Contain all the Needed Files?

The previous paragraph tests, that all tests pass in the tarball, but still check by inspection, that the tarball includes all necessary files. Files in the bin directory are potentially installed to /usr/bin, so be careful.

You may have a look an at MANIFEST.in which defines the files to be included in the tarball.

Do the Actual Release

Change version in sympy/__init__.py and doc/src/conf.py, e.g.

vim doc/src/conf.py  # Or your preferred text editor
vim sympy/__init__.py
git commit -a -m "SymPy 0.7.0 release"

Make sure you also change it everywhere else by doing git grep 0\.6\.7 (replacing 0\.6\.7 with the last release version).

Change the copyright year in README if needed. Note that this should be done at the beginning of each year.

Tag it, e.g.

git tag sympy-0.7.0

Prepare the release tarball and win32 installer (should work ok even on Linux)

./setup.py clean
./setup.py sdist
./setup.py bdist_wininst

Then run

rm -rf py3ksympy
./bin/use2to3
cd py3ksympy

and run the three above commands again.

Note: It's possible to make a 64-bit Windows installer if you are on Windows. See http://docs.python.org/distutils/builtdist.html#cross-compiling-on-windows and http://code.google.com/p/sympy/issues/detail?id=1235.

Compile the html documentation

cd doc
make clean
make html
cd _build
mv html sympy-docs-html-0.7.0
zip -9lr sympy-docs-html-0.7.0.zip sympy-docs-html-0.7.0

Do the same thing for the Python 3 docs that are generated by use2to3.

And put dist/sympy-0.7.0.tar.gz, dist/sympy-0.7.0.win32.exe and sympy-docs-html-0.7.0.zip on the website. Note that depending on what operating system you run the above in, you may need to rename the Windows binary to sympy-0.7.0.win32.exe (as mentioned above, even if you compile it in Linux or Mac OS X, it should still work in Windows).

Upload these files to http://code.google.com/p/sympy/downloads/ Use the files from previous versions as a guide. Be sure to remove the "Featured" tag from the files from the previous release.

Change version to 0.7.0-git, and start new development cycle

vim sympy/__init__.py
git ci -a -m 'Start of 0.7.1 development cycle'

Don't forget to finally push you changes back to the main repository:

git push [email protected]:sympy/sympy.git master

Get List of Authors who Contributed to the Release

Use git log sympy-0.6.7.. --format="%aN" --reverse | sort -u to get the list of contributors (see this page).

This is a good time to update the .mailmap file, so that there are no duplicates. You may need to email the people in question concerning the preferred spelling of their names/preferred email addresses, though you should generally prefer real email address to auto-generated addresses (like devnull@localhost or john@example.(None)), and names with accents to names without accents. This should also be in sync with the AUTHORS file. If you can't contact someone about the preferred email or name spelling, just use the version in the AUTHORS file. Note that when you update the .mailmap file, you should use the command git log --format="%aN <%aE>" | sort -u, which will show the whole log and include email addresses.

Make .mailmap completely up-to-date, and then it will be easy to find if there are any people missing from AUTHORS and doc/src/aboutus.txt. There are five people in AUTHORS who are not in the git history (they are marked with a *), and one person in the git history who is not in AUTHORS (by request). Therefore, the number of people in AUTHORS should be exactly four more than from git log --format="%aN" --reverse | sort -u | wc -l. Otherwise, people are missing, and they should be added before the release. Missing people in AUTHORS/aboutus should be added in the order of the first contribution. The easiest way to determine this for a missing person is to find the first commit that of that person (or technically, the first merge commit that put it in master), check that commit out, and look at the state of AUTHORS from that point in history.

See also the comments at the top of the AUTHORS and .mailmap.

Sort the authors everywhere by last name. Note that we decided to do this, instead of sorting by number commits or number of lines changed for fairness purposes (for example, if someone updates mpmath, they will unfairly have a larger count of line's changed; this can also happen, e.g., if someone moves some files around). You can change the above bash command to use sort -u -k 2 which sorts on the second "field" in the list, which should be the last name.

This list should be included at the bottom of the release notes. Mark anyone who contributed for the first time for this release with a * (you can get this list by running git diff sympy-0.6.7..sympy-0.7.0 -- AUTHORS).

Building a Mac OS X Installer Package

Note, this relies on https://github.com/sympy/sympy/pull/748

A Mac OS X PackageMaker project file can be found under the data/OS X Package directory. It references the isympy shell, its man page, and the sympy subdirectory. To build a new Installer package, open the project file and hit 'Build'. The resulting file can be distributed for Macs running OS X version 10.5 "Leopard" and above.

Building Package for MacPorts

Note, this relies on https://github.com/sympy/sympy/pull/830

Portfile can be found under the data/Macports. You have to:

  1. Bump version number in Portfile's version field
  2. Provide correct checksums of .tar.gz file (md5, sha1, rmd160). You can check them by running md5 sympy-x.x.x.tar.gz, openssl sha1 sympy-x.x.x.tar.gz, openssl sympy-x.x.x3.tar.gz
  3. Submit ticket to https://trac.macports.org/newticket

##How to Make New 64-bit Release of SymPy for Windows

The Windows SymPy binary("sympy-0.7.1.win32.exe") doesn't work, when the user using 64-bit Python release.

When try to install SymPy he get an error message: "No Python information found in the registry".

That's because "python-2.7.2.amd64.msi" installer put his regystry file in:

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.6\InstallPath]

but "sympy-0.7.1.win32.exe" binary search it in:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python\PythonCore\2.6\InstallPath]
and
[HKEY_CURRENT_USER\Software\Python\PythonCore]

So the user need to install "sympy-0.7.1.win-amd64.exe" witch is the 64-bit version of SymPy

###To Make New 64-bit Installer of SymPy We Need to Have Windows Machine and Then:

cd /path/to/SymPy
python setup.py build --plat-name=win-amd64 bdist_wininst

That will create "sympy-0.7.1.win-amd64.exe" in dist directory.

You can't create "sympy-0.7.1.win-amd64.exe" in other machine(linux,mac), because binary creator in them doesn't support "--plat-name" attribute. More information:

http://docs.python.org/distutils/builtdist.html#cross-compiling-on-windows

Sites to Update

If you don't have privileges to write to any of these, ask Aaron, and he will either give you write access or do it himself.

pypi

$ git co sympy-0.7.0
$ python setup.py register
$ python setup.py sdist upload

You may have to fix the uploaded file by hand over the web, so that it has the same md5sum as the one on googlecode. See also http://guide.python-distribute.org/contributing.html.

Make sure it is installable using easy_install and pip. For the documentation to upload for http://packages.python.org/sympy, the zip file you created above won't work, because it is a zip of the directory, and it wants an index.html at the root level. I was able to fix it by selecting everything in the sympy-0.7.0-docs-html directory in the Finder in Mac OS X and right clicking and choosing "Compress", and uploading the Archive.zip file that created. No doubt this could also be done with the command line (if you know how, please update the zip command above to do it).

Release Notes

If you haven't done it already, write up the release notes at Release-Notes-for-0.7.0, replacing "0.7.0" with this version. Use the notes from pervious versions as a guide. Go through the git log and include all important changes from the user perspective (there's no need to document changes that only affect internals). There's no need to be exhaustive against every single change, as there always exists the git log.

It's generally a good idea to enlist the people who made the various changes to write that section of the release notes, especially if you aren't familiar with that code or change.

Put the most important changes up front (at the top). This includes major new functionality and any backwards compatibility breaks.

Other Things

  • Merge the 0.7.0 branch back with master.
  • Delete the 0.7.0 branch.
  • Create a Debian package
  • Create a SymPy spkg Sage package. See http://code.google.com/p/sympy/wiki/SymPyspkg, which is kind of outdated. You can get a more up-to-date guide from the current Sage spkg. See http://trac.sagemath.org/sage_trac/ticket/11560#comment:8
  • Send an email to the list. Include a copy of the release notes and the list of people who contributed to the release.
  • Make sure that there are milestone labels in the issue tracker for at least two versions in the future. So if 0.7.0 is the most recently released version, make sure there are Milestone-release0.7.1 and Milestone-release0.7.2 labels.
  • Make sure any issues in the issue tracker that were fixed by the release are closed (this should not be done until the 0.7.0 branch is merged with master).

GCI Task :: Package the latest version of SymPy for the major distribution : OpenSUSE

What have I done:

  • I used SUSE Studio (because using it will become very easy)
  • I used SymPy 0.7.1 (and add the installer for linux at /sympy-installer directory
  • I've included Python there, so you just need to install the SymPy
  • I've built the package in 2 formats: Live CD / DVD (.iso) and VM Ware / VirtualBox / KVM (.vmdk)

That way is easier, and can be done by anyone.

Add a custom footer
Add a custom sidebar
Clone this wiki locally