This is the eighth article in the Flask TodoMVC tutorial, a series that creates a Backbone.js backend with Flask for the TodoMVC app. In this article, we reorganize our app into a package and introduce Flask Blueprints and application factories to assist in modularization.

Previous articles in this series:

1. Getting Started
2. Server Side Backbone Sync
3. Dataset Persistence
4. Custom Configuration
5. Testing Todo API
6. SQLAlchemy Persistence
7. User Login

We will begin with where we left off in the previous article. If you would like to follow along, but do not have the code, it is available on GitHub.

# Optional, use your own code if you followed the previous article
$ git clone https://github.com/kevinbeaty/flask-todomvc
$ cd flask-todomvc
$ git checkout -b modular login

Let's get started.

Introduction

Up to this point we've kept things simple and built most of our app withing a single file. As your app gets bigger, this can get out of hand. Arguably, we've already reached this point with server.py. That single file initializes our Flask app and two extensions, creates SQLAlchemy models for users, roles and todo items, and maps routes to the index and todo API. We are going to spend some time cleaning up our app.

Since we are going to moving a lot of code around, let's start with a roadmap of the file structure of where we are now, and where we will be at the end of this article. If you get lost, remember the code is available on GitHub from the article beginning to end.

Currently, our app hierarchy looks like this:

flask-todomvc
├── config
│   ├── __init__.py
│   ├── default.py
│   └── testing.py
├── requirements.txt
├── server.py
├── static
│   └── ...
├── templates
│   └── index.html
└── tests.py

Most of our code is in server.py. We have a single tests.py file and a config package that has default settings and testing overrides. The static and templates directories are mostly stolen (borrowed?) from the Backbone.js TodoMVC example.

We will gradually transition to a structure that looks like this:

flask-todomvc
├── flask_todomvc
│   ├── __init__.py
│   ├── extensions.py
│   ├── factory.py
│   ├── index.py
│   ├── models.py
│   ├── settings.py
│   ├── static
│   │   └── ...
│   ├── templates
│   │   └── index.html
│   └── todos.py
├── server.py
└── tests
    ├── __init__.py
    ├── settings.py
    └── todos_tests.py

Most code currently in server.py will be moved to a flask_todomvc package. We will split out initialization for our Flask extensions and models into their own modules, routes in todos.py and index.py and the static and templates directories into the package. We will include default settings in the app package and testing overrides in a tests package. The app will be initialized using an application factory used by both server.py and todos_tests.py.

Sound good? Let's start by created the flask_todomvc package.

$ mkdir flask_todomvc
$ touch flask_todomvc/__init__.py

Easy enough. Now we'll start moving some code.

Extensions

We'll start with moving our SQLAlchemy extension to extensions.py.

# flask_todomvc/exensions.py
from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()

Nothing more than importing and initializing. Let's use this extension in server.py.

-from flask_sqlalchemy import SQLAlchemy
+from flask_todomvc.extensions import db

..

-db = SQLAlchemy(app)
+db.init_app(app)

We import the db from our new extensions module and call init_app. Most Flask extensions can be initialized with one or more apps after creation by passing the same arguments to init_app as you could in the constructor.

This is helpful not only for separation of app and extension initialization, but also allows multiple apps to make use of the same extension. Say, for example, that you had an app for frontend users with session based authentication and another app serving an API using OAuth. You could create separate apps while using the same SQLAlchemy extension and data model.

Since we are no longer initializing the SQLAlchemy extension with the app, we need to update our tests so drop_all is done within an app context.

# tests.py
def tearDown(self):
    with server.app.app_context():
        server.db.drop_all()

You may recall we called create_all in init_db within an app context in the previous article, so no changes are necessary there.

Run the tests and server.py. Everything should still work.

Models

Now that we've separated the db initialization, we can move all our models into models.py.

# flask_todomvc/models.py
from flask_security import RoleMixin, UserMixin
from .extensions import db


class Todo(db.Model):
    __tablename__ = 'todos'
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String)
    order = db.Column(db.Integer)
    completed = db.Column(db.Boolean)

    def to_json(self):
        return {
            "id": self.id,
            "title": self.title,
            "order": self.order,
            "completed": self.completed}

    def from_json(self, source):
        if 'title' in source:
            self.title = source['title']
        if 'order' in source:
            self.order = source['order']
        if 'completed' in source:
            self.completed = source['completed']

roles_users = db.Table(
    'roles_users',
    db.Column('user_id', db.Integer, db.ForeignKey('users.id')),
    db.Column('role_id', db.Integer, db.ForeignKey('roles.id')))


class Role(db.Model, RoleMixin):
    __tablename__ = 'roles'
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String, unique=True)
    description = db.Column(db.String)


class User(db.Model, UserMixin):
    __tablename__ = 'users'
    id = db.Column(db.Integer, primary_key=True)
    email = db.Column(db.String, unique=True)
    password = db.Column(db.String)
    active = db.Column(db.Boolean)
    roles = db.relationship(
        'Role', secondary=roles_users,
        backref=db.backref('users', lazy='dynamic'))

We import db using a relative import from the extensions module, import the mixins from Flask-Security and define the models. We didn't have to make any changes to our model definitions.

Update server.py to remove the mixin and model imports and import the models from our new module.

# server.py
from flask_todomvc.models import User, Role, Todo
# Remove model definitions and UserMixin and RoleMixin import

Run the tests and server again to make sure everything is still working.

Security

Next, initialize the Flask-Security extension in extensions.py

# flask_todomvc/extensions.py
from flask_security import Security
...
security = Security()

Now update server.py to import from the extensions module and call init_app.

-from flask_todomvc.extensions import db
+from flask_todomvc.extensions import db, security

...

 from flask_security import (
-    Security,
     SQLAlchemyUserDatastore,
     login_required)

-security = Security(app, user_datastore)
+security.init_app(app, user_datastore)

Nothing should be surprising here. It's the same as the definition and initialization of the SQLAlchemy extension above.

Run tests and server again. Everything should still work.

Settings

We'll now move the default configuration into the package and call it settings.py.

mv config/default.py flask_todomvc/settings.py

Import the new file from the package and initialize the app config.

+from flask_todomvc import settings
...
-app.config.from_object('config.default')
+app.config.from_object(settings)

Notice that from_object takes either a python object or path.

Todos blueprint

Take a look at the routes. All but index are for Backbone.js Todo API. We're going to extract these routes into their own module using a Flask Blueprint.

It looks something like this:

# flask_todomvc/todos.py
from .extensions import db
from .models import Todo
from flask import (
    Blueprint,
    jsonify,
    request)

bp = Blueprint('todos', __name__, url_prefix='/todos')


@bp.route('/', methods=['POST'])
def create():
    todo = Todo()
    todo.from_json(request.get_json())
    db.session.add(todo)
    db.session.commit()
    return _todo_response(todo)


@bp.route('/<int:id>')
def read(id):
    todo = Todo.query.get_or_404(id)
    return _todo_response(todo)


@bp.route('/<int:id>', methods=['PUT', 'PATCH'])
def update(id):
    todo = Todo.query.get_or_404(id)
    todo.from_json(request.get_json())
    db.session.commit()
    return _todo_response(todo)


@bp.route('/<int:id>', methods=['DELETE'])
def delete(id):
    Todo.query.filter_by(id=id).delete()
    db.session.commit()
    return jsonify()


def _todo_response(todo):
    return jsonify(**todo.to_json())

We import the db proxy and models from our new modules and define the routes almost exactly as we have before. The implementation of the methods did not change from what we extracted from server.py. Instead of registering the routes on the Flask app, we create a Blueprint object and register our routes with it.

A blueprint is a set of objects that can be registered with an application. It allows easy separation of routes (and potentially templates, static files or resources) into their own modules. The blueprint can be registered with the app when the app is created, as opposed to during module definition and can be used to circumvent circular imports often encountered when attempting to define Flask apps within a package. You can register routes with the app directly, as we've seen before, in addition to Blueprint registration, if so desired. I would recommend just always using blueprints, even for smaller apps. In my opinion, it leads to cleaner separation of code.

When you initialize a blueprint, you provide a name as the first argument. This is used to namespace the routes registered with the blueprint when using url_for or related API. Since the blueprint has the namespace 'todos', and included in it's own module, we rename our methods to remove the todo_ prefix.

Update templates/index.html to specify the URL to the create method on our new blueprint.

- app.todos.url = '{{ url_for("todo_create") }}';
+ app.todos.url = '{{ url_for("todos.create") }}';

When using blueprints, you specify the blueprint name, followed by a dot, and then the method registered to the blueprint. If you do not specify the blueprint name, it is assumed to be the current blueprint. If we rendered this template from the todos blueprint, for example, we could have specified the url as ".create". In this case, the template is rendered inside server.py so we need to qualify the method with the blueprint.

Like Flask applications, the __name__ argument is used to identify the containing folder for blueprint resources. Specifying __name__ within a package will configure Flask to look for resources within the same package where the blueprint module is defined. Later, we will move the templates and static directories into the package, but the todos blueprint does not currently require resources, so we can wait to move these until later.

Notice that all route registrations no longer require a /todos prefix. This is because we specified a url_prefix when creating the blueprint. This is handy to save some typing. The prefix can also be specified when registering a blueprint with the app.

So how do we register this blueprint? Simple:

# server.py
from flask_todomvc.todos import bp as todos
...
app.register_blueprint(todos)
# Remove all todo_ routes (leave index)

We import the blueprint from our module as todos and call register_blueprint after creating the Flask app. You could register the Blueprint with different URL prefixes, if you so desire.

Make sure to remove all routes that we moved to the blueprint from server.py, run your tests and start the app. Everything should still work as before.

Index blueprint

We still have a route for rendering the template within server.py. It's perfectly valid to register direct routes and blueprints on the app, but I do prefer to use blueprints exclusively, so let's go ahead and create another blueprint for the index.

# flask_todomvc/index.py

""" index.py """
from flask import Blueprint, render_template
from flask_security import login_required

from .models import Todo

bp = Blueprint('index', __name__)


@bp.route('/')
@login_required
def index():
    todos = Todo.query.all()
    todo_list = map(Todo.to_json, todos)
    return render_template(
        'index.html', todos=todo_list)

Nothing should be surprising here. We create a blueprint without a prefix and register the index route with it.

Remove the unnecessary imports and index route from server.py and register the new blueprint.

Templates and static

Next, move the templates and static directories into our package.

$ mv templates flask_todomvc
$ mv static flask_todomvc
$ mv todos.db flask_todomvc

We also move our todos.db into the package since we are using a relative path to specify the SQLALCHEMY_DATABASE_URI in our config files. Normally you would specify a path to a user or var directory outside your app. We'll leave it as is for now though.

App factory

Since we have made quite a few changes, here is server.py in it's entirety at this point.

# server.py
from flask import Flask

from flask_todomvc import settings
from flask_todomvc.extensions import db, security
from flask_todomvc.models import User, Role
from flask_todomvc.index import bp as index
from flask_todomvc.todos import bp as todos

from flask_security import SQLAlchemyUserDatastore
from flask_security.utils import encrypt_password

app = Flask(__name__, static_url_path='')

app.config.from_object(settings)
app.config.from_envvar('TODO_SETTINGS', silent=True)

db.init_app(app)
user_datastore = SQLAlchemyUserDatastore(db, User, Role)
security.init_app(app, user_datastore)

app.register_blueprint(index)
app.register_blueprint(todos)


def init_db():
    with app.app_context():
        db.create_all()
        if not User.query.first():
            user_datastore.create_user(
                email='kevin@example.com',
                password=encrypt_password('password'))
            db.session.commit()


if __name__ == '__main__':
    init_db()
    app.run(port=8000)

We've extracted nearly everything from server.py into our new package. Take note, that nothing we've extracted required access to the app. We were able to avoid direct access by using blueprints and delaying the initialization of the extensions until app creation.

Since the app creation is self contained, we can easily convert it to use the application factory pattern.

# flask_todomvc/factory.py
from flask import Flask

from . import settings
from .extensions import db, security
from .models import User, Role
from .index import bp as index
from .todos import bp as todos

from flask_security import SQLAlchemyUserDatastore
from flask_security.utils import encrypt_password


def create_app():
    app = Flask(__name__, static_url_path='')

    app.config.from_object(settings)
    app.config.from_envvar('TODO_SETTINGS', silent=True)

    db.init_app(app)
    user_datastore = SQLAlchemyUserDatastore(db, User, Role)
    security.init_app(app, user_datastore)

    app.register_blueprint(index)
    app.register_blueprint(todos)

    with app.app_context():
        db.create_all()
        if not User.query.first():
            user_datastore.create_user(
                email='kevin@example.com',
                password=encrypt_password('password'))
            db.session.commit()
    return app

We moved the creation of an app into create_app. We also immediately created an app context and initialized the database (the same code which was formerly init_db).

Let's modify server.py to make use of the factory.

""" server.py """

from flask_todomvc.factory import create_app
app = create_app()
app.run(port=8000)

That's it, in it's entirety. We've successfully moved our code into a package.

Start the server again to make sure everything still works. Then modify your tests to work with the factory.

-import server
+from flask_todomvc.extensions import db
+from flask_todomvc.factory import create_app


 class TodoTestCase(unittest.TestCase):

     def setUp(self):
-        self.client = server.app.test_client()
+        self.app = create_app()
+        self.client = self.app.test_client()
         self.order = 1
-        server.init_db()

     def tearDown(self):
-        with server.app.app_context():
-            server.db.drop_all()
+        with self.app.app_context():
+            db.drop_all()

     def test_config_settings(self):
-        config = server.app.config
+        config = self.app.config
         assert config['SQLALCHEMY_DATABASE_URI'] == \
             'sqlite:///test.db'
         assert config['TESTING']

We create an app with the factory during setup and set on the test instance. We no longer need to init_db because we now do that within create app. We also use the imported db extension during tear down. The configuration is still overridden with an environment variable. This is a little messy; we will address this later.

Tests

Now that we've converted our app to a package, what about our tests? Let's create a package for those as well.

$ mkdir tests
$ touch tests/__init__.py
$ mv tests.py tests/todos_tests.py

We'll also move our test settings into the new package.

$ mv config/testing.py tests/settings.py
$ rm -r config

After we move the settings, We no longer need the config package, so go ahead and remove it.

We're going to fix our tests to use our app factory, but before we do that, let's add a way to override the default settings without using an environment variable.

# flask_todomvc/factory.py
def create_app(priority_settings=None):
    app = Flask(__name__, static_url_path='')

    app.config.from_object(settings)
    app.config.from_envvar('TODO_SETTINGS', silent=True)
    app.config.from_object(priority_settings)
    ...

We added an optional keyword argument for priority settings that would override any default or settings from an environment variable. This demonstrates one benefit of the application factory pattern: you can customize app creation on initialization time.

Let's update our tests to use the factory to create an application with the test settings.

-import os
 import unittest
 import json
-from os import path
-
-base_path = path.dirname(path.realpath(__file__))
-cfg_path = path.join(base_path, 'config', 'testing.py')
-os.environ['TODO_SETTINGS'] = cfg_path

+from . import settings

     def setUp(self):
-        self.app = create_app()
+        self.app = create_app(
+            priority_settings=settings)
         self.client = self.app.test_client()
         self.order = 1
-
-
-if __name__ == '__main__':
-    unittest.main()

We remove the code to set the config file using an environment variable and instead import the settings from the tests package and pass as priority settings to the factory. We also removed the call to unnittest.main. We will instead use nose to run our tests.

$ pip install nose
$ nosetests

Since we named our tests package and module included the word tests, nose automatically found and ran the tests in the package. We can create more tests within the package and nose will find and run those as well without configuration.

Conclusion

In this article, we focused on modularization. We recognized that our single file app was a little cluttered and refactored it to cleaner modules within a package. We accomplished this by making use of blueprints and the app factory pattern. We reused this factory in a much leaner server.py and providing priority setting overrides when testing.

If you made it this far you should follow me on Twitter and GitHub.

The code is available on GitHub with tag modular or compared to previous article.