*********************************
How to Structure a Django Project
*********************************

Lots of Examples in the Wild
============================

There are many opinions among Django developers on how a project should be structure. There *is* more than one way to do
it. Here, we present the way *we* do it, and have done over the course of a decade of development.

First some definitions:

- *Project Root* is the top-most directory of the project where all project files and directories are stored. Most,
  though not all, of its contents will be under version control.
- *Source Code* refers to all of the Python code, Django apps, and related static and configuration files that come into
  use with ``runserver``.

Starting Point
==============

The most basic structure includes a ``source/`` directory in which the Django project is defined as ``main/``.

.. note::
    Why *source*? It makes the location of source code (defined above) obvious.

This may be accomplished with the following commands:

.. code-block:: bash

    mkdir -p project_name/source;
    cd project_name/source;
    django-admin startproject main .;

Our starting structure is therefore:

.. code-block:: text

    project_name
    `-- source
        |-- main
        |   |-- __init__.py
        |   |-- settings.py
        |   |-- urls.py
        |   `-- wsgi.py
        `-- manage.py

.. note::
    Why *main*? As a convention of Python, the word main indicates that this is the "main" thing; that is, the Django
    project where everything starts. Keeping the Django project files separate makes it clear where configuration and
    URLs are to be defined.

Location of Apps
================

We've found it beneficial to separate Django apps into three broad classes:

1. Contributing functionality. Located in `contrib/`, these apps (or libraries) provide *non-specific* resources that
   may be used across all other areas of the project. Contrib apps may be good candidates for eventual re-factoring into
   stand-alone apps that may be installed via pip.
2. Local functionality. Local apps provide functionality that is specific or core to the purpose of the project.
3. Shared functionality. Shared apps (or libraries) provide *specific* resources across all other areas of the project.
   One example is commonly used lookup (validation) data. While a contrib app is completely generic, and a local app is
   completely specific, a *shared* app is somewhere in between.

.. note::
    ``local/`` is the only directory we create for every project. Large projects often incoprorate ``shared/`` and
    sometimes ``contrib/``. Why bother with the separation? Creating contrib apps keeps the project moving without
    refactoring -- this can be done later. Shared functionality is anything that is re-used across local apps where it
    makes more sense to keep it separate rather than located within a local (and more specific) app, e.g. lookup models,
    configuration models, integrations, and so on.

Apps are therefore created like so:

.. code-block:: bash

    ./manage.py startapp contrib_app_name;
    mv contrib_app_name contrib/app_name;

    ./manage.py startapp local_app_name;
    mv local_app_name local/app_name;

    ./manage.py startapp shared_app_name;
    mv shared_app_name shared/app_name;

Note that the app's config will need to be updated, for example:

.. code-block:: python

    class DefaultConfig(AppConfig):
        name = "local.app_name"
        label = "app_name"
        verbose_name = _("App Name")

The Glue App
============

The "glue" app may be used to tie various elements of the project together by customizing CMS or UI functionality,
providing additional views, template overrides, and so on.

It's created with ``./manage.py startapp glue``.

.. note::
    We first started using the glue app when building web sites with Wagtail and then also adopted this approach for
    creating menus with SuperDjango UI. It is especially useful for Wagtail hooks, defining dashboard views, menus, and
    overriding templates. These days, we scarcely have a project that does not utilize a glue app. Why "glue"? Well,
    because it "glues" various resources together.

Theme and Static Files
======================

The ``static/`` directory contains project-specific static files.

The ``theme/`` directory contains the main HTML (and static) files used to deliver the look and feel of the project.
It may or may *not* be project specific.

Static files for the theme are structured in the same way as the ``static/`` directory.

We like to collect static files into a ``www/`` directory. Our settings therefore look like this:

.. code-block:: python

    # Static files (CSS, JavaScript, Images)
    # https://docs.djangoproject.com/en/3.0/howto/static-files/
    STATICFILES_DIRS = [
        os.path.join(BASE_DIR, "static"),
        os.path.join(BASE_DIR, "theme", "static"),
    ]
    STATIC_ROOT = os.path.join(BASE_DIR, "www", "assets")
    STATIC_URL = "/_assets/"

    MEDIA_ROOT = os.path.join(BASE_DIR, "www", "content")
    MEDIA_URL = "/_content/"

And in ``urls.py``:

.. code-block:: python

    if settings.DEBUG:
        urlpatterns += static(
            settings.STATIC_URL,
            document_root=settings.STATIC_ROOT
        )
        urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Supporting Files and Directories
================================

A number of supporting directories may also be included:

- The ``assets/`` directory contains the source files for any static files included in the project.
- The ``deploy/`` directory contains instructions, scripts, and other resources used to deploy the project.
- The ``docs/`` directory contains source files and build scripts for generating project documentation.
- In the development environment, the ``logs/`` directory may be created in project root. It should be excluded from source code and deployments.
- The ``provision/`` directory contains instructions and (possibly) scripts that may be used to provision deployment resources for the project.
- The ``tests/`` directory contains unit tests for the project.

.. note::
    Supporting directories are not source code, so they are located in project root.

Recommended Meta Files
----------------------

The following meta files should generally always be included:

- ``.gitignore``: Controls what is included in source code management.
- ``LICENSE.txt``: The license for the project, even if just a simple copyright statement for proprietary or closed-source licensing.
- ``README.markdown``: Basic information and documentation regarding the project.
- ``VERSION.txt``: The version/release identifier for the project.

.. note::
    Why files with extensions? A lot of projects (including third-party apps and libraries) simply have a LICENSE or
    VERSION file (for example). Adding the extensions makes the file type obvious to both humans and text editors.

Suggested Meta Files
--------------------

The following meta files are recommended:

- ``.coveragerc``: Coverage options for testing.
- ``.status``: The current development status of the project; suggested to be one of ``active``, ``hold``, ``unknown``, ``maintenance``, ``closed``. This is useful (among other things) for scanning a directory of projects to determine the status of each.
- ``DESCRIPTION.txt``: A brief description of the project.
- ``pytest.ini``: Configuration options for `testing with pytest`_. (See also `the pytest-django plugin`_.)
- ``Makefile``: Various build options codified in make format.
- ``meta.ini``: Meta data regarding the project.

.. _the pytest-django plugin: https://pytest-django.readthedocs.io/en/latest/
.. _testing with pytest: https://pytest.org/en/latest/

.. note::
    We use the .status and meta.ini files in conjunction with command line tools for managing projects.

Including SuperDjango in Source
===============================

Finally, a word about how we work with SuperDjango: As of this writing, the project is still under heavy development in all areas and components. For this reason, there is no ``setup.py`` file. The choices are to install it manually into the environment, add it to the Python path, or create a symlink. We opt for the symlink option.

First, we clone the superdjango project in ``$PROJECT_HOME``. Then we create a symlink in project root (``source/``) to ``$PROJECT_HOME/superdjango/superdjango/``. In PyCharm, the ``source/`` directory is marked as "Sources Root" which automatically adds superdjango to the Python path.

We find that this is a convenient means of working with both source code for a project and superdjango at the same time.

.. note::
    What about deployment? Our deployment scripts utilize rsync to push the contents of project root to the target environment. We use the ``--copy-links`` option to ensure that superdjango (and any other symlinks) are materialized as they are copied to the server.

The Final Structure
===================

The final structure (with some example apps) now looks something like this:

.. code-block:: text

    project_name
    |-- assets
    |-- deploy
    |-- docs
    |-- logs
    |-- provision
    |-- source
    |   |-- contrib
    |   |   `-- __init__.py
    |   |-- glue
    |   |   |-- templates
    |   |   |   `-- glue
    |   |   |       `-- dashboard.html
    |   |   |-- __init__.py
    |   |   |-- ui.py
    |   |   `-- views.py
    |   |-- local
    |   |   |-- projects
    |   |   |   |-- migrations
    |   |   |   |   `-- __init__.py
    |   |   |   |-- __init__.py
    |   |   |   |-- apps.py
    |   |   |   |-- models.py
    |   |   |   |-- ui.py
    |   |   |   `-- views.py
    |   |   |-- todos
    |   |   |   |-- migrations
    |   |   |   |   `-- __init__.py
    |   |   |   |-- __init__.py
    |   |   |   |-- models.py
    |   |   |   |-- ui.py
    |   |   |   `-- views.py
    |   |   `-- __init__.py
    |   |-- main
    |   |   |-- __init__.py
    |   |   |-- settings.py
    |   |   |-- urls.py
    |   |   `-- wsgi.py
    |   |-- shared
    |   |   |-- integrations
    |   |   |   |-- migrations
    |   |   |   |   `-- __init__.py
    |   |   |   |-- __init__.py
    |   |   |   |-- models.py
    |   |   |   |-- ui.py
    |   |   |   `-- views.py
    |   |   |-- lookups
    |   |   |   |-- migrations
    |   |   |   |   `-- __init__.py
    |   |   |   |-- __init__.py
    |   |   |   |-- models.py
    |   |   |   |-- ui.py
    |   |   |   `-- views.py
    |   |   `-- __init__.py
    |   |-- static
    |   |   |-- css
    |   |   |   `-- custom.css
    |   |   |-- fonts
    |   |   |-- images
    |   |   `-- js
    |   |       `-- custom.js
    |   |-- superdjango -> $PROJECT_HOME/superdjango/superdjango
    |   |-- theme
    |   |   |-- includes
    |   |   |   |-- breadcrumbs.html
    |   |   |   |-- footer.html
    |   |   |   |-- menu.html
    |   |   |   |-- messages.html
    |   |   |   |-- navbar.html
    |   |   |   `-- submenu.html
    |   |   |-- static
    |   |   |   |-- css
    |   |   |   |-- fonts
    |   |   |   |-- images
    |   |   |   `-- js
    |   |   |-- 403.html
    |   |   |-- 404.html
    |   |   |-- 500.html
    |   |   |-- 503.html
    |   |   |-- base.html
    |   |   |-- sidebar_left.html
    |   |   |-- sidebar_right.html
    |   |   `-- three_columns.html
    |   |-- www
    |   `-- manage.py
    |-- tests
    |-- DESCRIPTION.txt
    |-- LICENSE.txt
    |-- Makefile
    |-- README.markdown
    |-- VERSION.txt
    |-- meta.ini
    `-- pytest.ini