Clean install with python3

1 files changed, 212 insertions, 74 deletions

diff --git a/README.txt b/README.txt
index 1ad17bb..c6a5c79 100644
--- a/README.txt
+++ b/README.txt
@@ -1,59 +1,215 @@
-Troggle is an application for caving expedition data management, originally created for use on Cambridge University Caving Club (CUCC)expeditions and licensed under the GNU Lesser General Public License.
+Troggle is an application for caving expedition data management, 
+originally created for use on Cambridge University Caving Club (CUCC)expeditions 
+and licensed under the GNU Lesser General Public License.
 
-Troggle has been forked into two projects. The original one is maintained by Aron Curtis and is used for Erebus caves. The CUCC variant uses files as the definitive data, not the database and lives at expo.survex.com/troggle.
+Troggle has been forked into two projects. The original one is maintained by Aron Curtis 
+and is used for Erebus caves. The CUCC variant uses files as the definitive data, 
+not the database and lives at http://expo.survex.com/repositories/troggle/.git/
 
 Troggle setup
-==========
+=============
+1. git clone troggle into correct directory structure
+2. install pip, django & patch django
+3. configure django to recognise troggle 
+
+Setting up directories
+----------------------
+- create a directory in which you also have the loser/, expoweb/ and drawings/ repositories
+- git clone (see below) the current latest troggle from the 'python3' git branch into 
+  a folder called 'troggle'
+- all the 4 repos should now be siblings, and also siblings with expofiles/
+- if you have those repos elsewhere, set up symlinks in the directory above troggle
+  so that troggle thinks they are siblings
 
-Python, Django, and Database setup
+Troggle itself
+-------------
+Choose the directory where you will keep troggle, (which must be named "troggle")
+and git clone troggle into it using the following command:
+
+git clone git://expo.survex.com/troggle
+or more reliably
+git clone ssh://expo@expo.survex.com/home/expo/troggle
+but you will have to set up the ssh key-exchange with the server to make that work.
+Key-exchange instructions: http://expo.survex.com/handbook/computing/keyexchange.html
+
+If you want to work on the source code and be able to commit, your account will need to be 
+added to the troggle project members list. Contact wookey  at wookware dot org to get this set up.
+
+Next, you need to fill in your local settings. Copy localsettingsWSL.py 
+to a new file called localsettings.py and edit it and settings.py to match
+your machine's file locations. 
+Follow the instructions contained in the file to fill out your settings.
+
+{ *TO BE FIXED*
+  The localsettings-expo-live.py is the python2.7 settings for the server.
+  These are all very out of date and need fixing:
+  localsettingsubuntu.py
+  localsettingsdocker.py
+  localsettingswindows.py
+  localsettingspotatohut.py
+}
+
+Python3, Django, and Database setup
 -----------------------------------
-Troggle requires Django 1.4 or greater, and any version of Python that works with it.
-It is currently (Feb.2020) on django 1.7.11  (1.7.11-1+deb8u5).
-Install Django with the following command:
-
-sudo apt install python-django  (on debian/ubuntu) -- does not work now as we need specific version
-
-requirements.txt:
-Django==1.7.11
-django-registration==2.1.2
-mysql
-#imagekit
-#django-imagekit
-Image
-django-tinymce==2.7.0
-smartencoding
-unidecode
-
-Install like this:
-sudo apt install pip       # does not work on Ubuntu 20.04 for python 2.7. Have to install from source. Use 18.04
-pip install django==1.7
-pip install django-tinymce==2.0.1
-sudo apt install libfreetype6-dev
-pip install django-registration==2.0
-pip install unidecode
-pip install --no-cache-dir pillow==2.7.0 # fails horribly on installing Ubuntu 20.04 
-pip install --no-cache-dir pillow # installs on Ubuntu 20.04 , don't know if it works though
+Troggle with python3 requires Django 1.7 .
+It is currently (May 2020) on django 1.7.11  (1.7.11-1+deb8u5).
 
-If you want to use MySQL or Postgresql, download and install them. 
-However, you can also use Django with Sqlite3, which is included in Python and thus requires no extra installation.
-pip install pygraphviz
-apt install survex
+Read this: https://docs.djangoproject.com/en/3.0/topics/install/
 
-pip install django-extensions
-pip install pygraphviz # fails to install
-pip install pyparsing pydot # installs fine
-django extension graph_models # https://django-extensions.readthedocs.io/en/latest/graph_models.html
+We are installing with python3.
+[ If you want to use python2.7 you presumably can work it out for yourself.
+You should look at past revisions of this file in git to see what is required.
+Also : https://linuxize.com/post/how-to-install-pip-on-ubuntu-20.04/ ]
+
+Install Django using pip, not with apt.
+Your Linux installation almost certainly already includes python3 and pip3 but
+in case it doesn't install those like this:
+
+$ sudo apt update
+$ sudo apt dist-upgrade
+$ sudo apt install python3
+$ sudo apt install python3-pip
+
+Now install django etc.
+$ sudo pip3 install -r requirements.txt
+
+where requirements.txt is:
+Django==1.7
+django-extensions==2.2.9
+django-registration==2.0
+django-tinymce==2.0.1
+Pillow==7.1.2
+six==1.14.0
+Unidecode==1.1.1
+
+if you don't use sudo it will install them all in ~/.local/
+and so will only be available for you, not everyone; and
+the paths won't work to find troggle properly.
 
+six and Unidecode handle some of the python2-3 conversions
+and Pillow is an image handling package used to make
+the prospecting map. tinymce is the wysiwyg in-browser
+editor.
+
+$ pip3 list
+will list all the pip python packages installed.
+
+[NB we should test whether later verisons of tinymce work.]
+
+venv option
+-----------
 Or use a python3 virtual environment: (python3.5 not later)
 $ cd troggle
 $ cd ..
 $ python3.5 -m venv pyth35d2
 (creates folder with virtual env)
-cd pyth35d2
-bin/activate
-(now install everything - not working yet..)
+$ cd pyth35d2
+$ source bin/activate
+(now install everything )
 $ pip install -r requirements.txt
 
+This works if you need a specific version of python3
+but doesn't really work with python2.7 as you also need to obtain
+an elderly version of pip that will work and use and older virtual
+environment utility
+
+Patching the django installation
+--------------------------------
+Since django 1.7 out of update-support, and since python3 has progressed against
+the 2015 version that django1.7 is expecting, you need to patch the django
+installation after installing. In the troggle folder you will find
+django-patch/html_parser.py
+copy this over the installed version of django on your machine:
+
+$ cd troggle
+$ sudo cp django-patch/html_parser.py /usr/local/lib/python3.8/dist-packages/django/utils/
+or
+$ cp django-patch/html_parser.py ~/.local/lib/python3.8/site-packages/django/utils/
+if you didn't use sudo when installing everything using pip.
+
+that's it.
+
+Testing the django installation
+-------------------------------
+Test things by running these commands:
+$ django-admin --version
+$ django-admin 
+It should show a list of commands and complain:
+"..only Django core commands are listed as settings are not properly configured" 
+
+If you get an error when running
+$ django-admin
+Then run django-admin like this:
+python /usr/local/lib/python3.8/dist-packages/django/bin/django-admin.py
+which should fix the paths. From now on you should be able to run 
+$ django-admin 
+from within any folder on your machine.
+
+Now do
+$ cd troggle
+$ python manage.py
+
+You should see the same list of commands that you saw with django-admin but wth 
+a lot of extra ones and no complaints. This means it is reading at least some of your
+settings correctly.
+
+If you get an error your python sys.path is probably not set correctly yet. Do
+$ python -m site
+when you are in your troggle directory to see the list of paths python looks 
+for when it is searching for packages (both django and troggle). 
+Ensure that the path to the troggle/ directory is in the list.
+It should be at the top, which is where the current working directory is.
+
+Registering troggle as a django application
+-------------------------------------------
+In your troggle directory run
+$ django-admin 
+and check you got the same output as before.
+
+Then try 
+$ python manage.py
+It may not work. But if it does it will
+now show a superset of the previous output: the [django] commands
+available but also other options. Run
+$ python manage.py check
+$ python manage.py diffsettings
+This last one shows everything set in global settings, settings and localsettings.
+Anything different from global settings  (django built-in) has '###' appended.
+
+$ python manage.py test -v 2
+Tests that it can create a database from all the model files.
+
+$ python manage.py migrate
+Tests the uptodateness of your sqlite database.
+$ python manage.py help migrate
+explains what this does and gives extra command line options.
+
+If you got an error traceback with
+$ python manage.py
+then the settings registration of troggle with django is incomplete.
+Delete all your cached .pyc files and try again.
+You probably have a mistake in your settings.py or localsettings.py files.
+
+Setting up survex
+-----------------
+You need to have survex installed as the command line tools 'cavern' and '3dtopos' are
+used as part of the survex import process.
+$ sudo apt install survex
+
+Setting up tables and importing survey data
+-------------------------------------------
+Run 
+$ sudo python databaseReset.py
+from the troggle directory will give you instructions.
+
+[ NB Adding a new year/expedition requires adding a column to the
+folk/folk.csv table - a year doesn't exist until that is done.]
+
+Database
+--------
+If you want to use MySQL or Postgresql, download and install them. 
+However, you can also use Django with sqlite3, which is included in Python and thus requires no extra installation.
+
 MariaDB database
 ----------------
 Start it up with
@@ -74,42 +230,16 @@ $ sudo systemctl restart mysql.service
 $ sudo systemctl stop mysql.service
 $ sudo systemctl start mysql.service
 
-Troggle itself
--------------
-Choose a directory where you will keep troggle, and git clone Troggle into it using the following command:
-
-git clone git://expo.survex.com/troggle
-or more reliably
-git clone ssh://expo@expo.survex.com/home/expo/troggle
-
-
-If you want to work on the source code and be able to commit, your account will need to be added to the troggle project members list. Contact wookey  at wookware dot org to get this set up.
-
-Next, you need to fill in your local settings. Copy either localsettingsubuntu.py or localsettingsserver.py to a new file called localsettings.py. Follow the instructions contained in the file to fill out your settings.
-
-Setting up survex
------------------
-You need to have survex installed as the command line 'cavern' is used as part of the survex
-import process.
-
-Setting up tables and importing legacy data
-------------------------------------------
-Run "sudo python databaseReset.py reset" from the troggle directory.
-
-
-Once troggle is running, you can also log in and then go to "Import / export" data under "admin" on the menu.
-
-Adding a new year/expedition requires adding a column to the
-folk/folk.csv table - a year doesn't exist until that is done.
-
-
 Running a Troggle server
 ------------------------
-For high volume use, Troggle should be run using a web server like apache. However, a quick way to get started is to use the development server built into Django. This is limited though: directory 
-redirection needs apache.
-
-To do this, run "python manage.py runserver" from the troggle directory.
+For high volume use, Troggle should be run using a web server like apache. 
+However, a quick way to get started is to use the development server built into Django. 
+This is limited though: directory redirection needs apache.
 
+To do this, run 
+$ python manage.py runserver 8000 -v 3
+from the troggle directory. This runs it on port 8000 so you see the website
+at http://localhost:8000/
 
 Running a Troggle server with Apache
 ------------------------------------
@@ -140,3 +270,11 @@ any changed files:
 
 apache2ctl stop
 apache2ctl start
+
+Experimental additions
+----------------------
+These are modern tools which help us document how troggle works.
+
+pip install pygraphviz 
+pip install pyparsing pydot # installs fine
+django extension graph_models # https://django-extensions.readthedocs.io/en/latest/graph_models.html
\ No newline at end of file