jamesturk / django-markupfield

📑 a MarkupField for Django

django-markupfield

An implementation of a custom MarkupField for Django. A MarkupField is in essence a TextField with an associated markup type. The field also caches its rendered value on the assumption that disk space is cheaper than CPU cycles in a web application.

Installation

The recommended way to install django-markupfield is with pip

It is not necessary to add 'markupfield' to your INSTALLED_APPS, it merely needs to be on your PYTHONPATH. However, to use titled markup you either add 'markupfield' to your INSTALLED_APPS or add the corresponding translations to your project translation.

Requirements

Requires Django >= 2.2 and 3.6+

• 1.5.x is the last release to officially support Django < 2.2 or Python 2.7
• 1.4.x is the last release to officially support Django < 1.11
• 1.3.x is the last release to officially support Django 1.4 or Python 3.3

Settings

To best make use of MarkupField you should define the MARKUP_FIELD_TYPES setting, a mapping of strings to callables that 'render' a markup type:

import markdown
from docutils.core import publish_parts

def render_rest(markup):
    parts = publish_parts(source=markup, writer_name="html4css1")
    return parts["fragment"]

MARKUP_FIELD_TYPES = (
    ('markdown', markdown.markdown),
    ('ReST', render_rest),
)

If you do not define a MARKUP_FIELD_TYPES then one is provided with the following markup types available:

html:

allows HTML, potentially unsafe

plain:

plain text markup, calls urlize and replaces text with linebreaks

markdown:

default markdown renderer (only if markdown is installed)

restructuredtext:

default ReST renderer (only if docutils is installed)

It is also possible to override MARKUP_FIELD_TYPES on a per-field basis by passing the markup_choices option to a MarkupField in your model declaration.

Usage

Using MarkupField is relatively easy, it can be used in any model definition:

from django.db import models
from markupfield.fields import MarkupField

class Article(models.Model):
    title = models.CharField(max_length=100)
    slug = models.SlugField(max_length=100)
    body = MarkupField()

Article objects can then be created with any markup type defined in MARKUP_FIELD_TYPES:

Article.objects.create(title='some article', slug='some-article',
                       body='*fancy*', body_markup_type='markdown')

You will notice that a field named body_markup_type exists that you did not declare, MarkupField actually creates two extra fields here body_markup_type and _body_rendered. These fields are always named according to the name of the declared MarkupField.

Arguments

MarkupField also takes three optional arguments. Either default_markup_type and markup_type arguments may be specified but not both.

default_markup_type:

Set a markup_type that the field will default to if one is not specified. It is still possible to edit the markup type attribute and it will appear by default in ModelForms.

markup_type:

Set markup type that the field will always use, editable=False is set on the hidden field so it is not shown in ModelForms.

markup_choices:

A replacement list of markup choices to be used in lieu of MARKUP_FIELD_TYPES on a per-field basis.

escape_html:

A flag (False by default) indicating that the input should be regarded as untrusted and as such will be run through Django's escape filter.

Examples

MarkupField that will default to using markdown but allow the user a choice:

MarkupField(default_markup_type='markdown')

MarkupField that will use ReST and not provide a choice on forms:

MarkupField(markup_type='restructuredtext')

MarkupField that will use a custom set of renderers:

CUSTOM_RENDERERS = (
    ('markdown', markdown.markdown),
    ('wiki', my_wiki_render_func)
)
MarkupField(markup_choices=CUSTOM_RENDERERS)

Note

When using markdown, be sure to use markdown.markdown and not the markdown.Markdown class, the class requires an explicit reset to function properly in some cases. (See [issue #40](#40) for details.)

Accessing a MarkupField on a model

When accessing an attribute of a model that was declared as a MarkupField a special Markup object is returned. The Markup object has three parameters:

raw:

The unrendered markup.

markup_type:

The markup type.

rendered:

The rendered HTML version of raw, this attribute is read-only.

This object has a __unicode__ method that calls django.utils.safestring.mark_safe on rendered allowing MarkupField objects to appear in templates as their rendered selfs without any template tag or having to access rendered directly.

Assuming the Article model above:

>>> a = Article.objects.all()[0]
>>> a.body.raw
u'*fancy*'
>>> a.body.markup_type
u'markdown'
>>> a.body.rendered
u'<p><em>fancy</em></p>'
>>> print unicode(a.body)
<p><em>fancy</em></p>

Assignment to a.body is equivalent to assignment to a.body.raw and assignment to a.body_markup_type is equivalent to assignment to a.body.markup_type.

Important

Keeping in mind that body is MarkupField instance is particullary important with default or default_if_none filter for model that could be blank. If body's rendered is None or empty string ("") these filters will not evaluate body as falsy to display default text:

{{ a.body|default:"<missing body>" }}

That's because body is regular non-None MarkupField instance. To let default or default_if_none filters to work evaluate rendered MarkupField attribute instead. To prevent escaping HTML for the case rendered is truethy, finish chain with safe filter:

{{ a.body.rendered|default:"<missing body>"|safe }}

Note

a.body.rendered is only updated when a.save() is called

django-markupfield's Issues

Tables not rendering

I've tried a number of different ways of creating tables and none of them render. A few examples are (plus many other variations):

| field 1 | field 2 |
| field 3 | field 4 |


| field 1 | field 2 |
|:---|:---|
| field 3 | field 4 |


field 1 | field 2
field 3 | field 4

MarkupField causes error in syncdb (abstract model inheritance).

Hi I have a model A which is an abstract baseclass for model B. Model A has a MarkupField called content. This scenario raises the following error:

Traceback (most recent call last):
  File "bin/django", line 20, in <module>
    djangorecipe.manage.main('website.settings')
  File "/home/phxx/projects/gregor/trunk/eggs/djangorecipe-0.20-py2.6.egg/djangorecipe/manage.py", line 16, in main
    management.execute_manager(mod)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/core/management/__init__.py", line 362, in execute_manager
    utility.execute()
  File "/home/phxx/projects/gregor/trunk/parts/django/django/core/management/__init__.py", line 303, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/core/management/base.py", line 195, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/core/management/base.py", line 222, in execute
    output = self.handle(*args, **options)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/core/management/commands/syncdb.py", line 78, in handle_noargs
    cursor.execute(statement)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/db/backends/util.py", line 19, in execute
    return self.cursor.execute(sql, params)
  File "/home/phxx/projects/gregor/trunk/parts/django/django/db/backends/sqlite3/base.py", line 193, in execute
    return Database.Cursor.execute(self, query, params)
pysqlite2.dbapi2.OperationalError: duplicate column name: content_markup_type

This error will not occur if I move the content field into model B.

Please make new release for Django 3.0 compatibility

I see that Django 3.0 compatibility has been already addressed by 37307f9, can you please release this to PyPI?

Need help with set up

Hello,

I'd like to use MarkupField with ReST in admin app when I want to add new blog post. Unfortunately I'm not able to run it properly. My current config looks like this:

settings.py

from docutils.core import publish_parts

def render_rest(markup):
    parts = publish_parts(source=markup, writer_name="html4css1")
    return parts["fragment"]

MARKUP_FIELD_TYPES = (
    ('ReST', render_rest),
)

models.py

class Blog(models.Model):
    ...
    mdf = MarkupField(null=True, blank=True)
    ...

I successfuly migrated (using South) and I can see this field in django admin when I want to add Blog object.

But when I try to type e.g.:

====================
Heading 1
====================

Heading 2
-----------------

Ok what now **bold text** and here *italic* and try ``code``.

`THis link is unfortunately relative <www.google.com>`_

In final page I end up with no headings (they are not just hidden - when I run developer tools, they are just not rendered), I can't manage to make link absolute (now I end up mydomain.com/blog/www.google.com and code is represented as <tt class="docutils literal">code</tt> - but that is rendered as normal text - do I need to edit manually these classes in style.css?

Thank you for you help.

indexing MarkupFields with wagtail & elasticsearch

We are running into issues when indexing a Django model with MarkupFields into Elasticsearch using wagtail. The Markup object returned when accessing the field property can't be serialized to Elasticsearch.

There's a few ways around this but I'd like to avoid having to create additional methods (e.g., get_raw_foo). I think the easiest way would be to add a get_searchable_content method to the Markup class that returns the raw source or to somehow make the MarkupField more open to extension.. I can't extend the MarkupField class directly since it gets replaced with the Markup class.

Thoughts? And thank you for contributing & maintaining this!

django-markupfield raises deprecation error in Django 1.3

Error message:

markupfield/fields.py:80: DeprecationWarning: A Field class whose get_db_prep_value method hasn't been updated to take connection and prepared arguments.

(class MarkupField(models.TextField):

{% if model.field %} and {{ model.field|default }} don't work

this should be fixed, as it is surprising

AttributeError: 'NoneType' object has no attribute 'markup_type'

Attempting to create a nullable MarkupField whose default is None:

class Person(models.Model):

    name = models.CharField(max_length=50)
    biography  = MarkupField( markup_type='markdown', blank=True, null=True, default=None)
    ...

However, when attempting to create a person object:

>>> p = Person.objects.create(**{"name": "John Doe"})
File "/lib/python2.7/site-packages/markupfield/fields.py", line 146, in pre_save
    if value.markup_type not in self.markup_choices_list:
AttributeError: 'NoneType' object has no attribute 'markup_type'

An alternate definition works:

class Person(models.Model):

    name = models.CharField(max_length=50)
    biography  = MarkupField( markup_type='markdown', blank=True, null=True, default="")
    ...

But I'd prefer not to populate my database with empty strings when they could just be NULLs.

'unicode' object has no attribute 'raw' with Django REST-Framework

I'm trying to have the Django REST Framework serialize a MarkupField, however there appears to be an error during serialization when calling the value_to_string method. On line 156 of fields.py

def value_to_string(self, obj):
    value = self._get_val_from_obj(obj)
    return value.raw

When obj is None, the _get_val_from_obj method appears to return u'', which does not have a raw properterty to access.

Proposed fix:

def value_to_string(self, obj):
    if obj is None: return u''
    value = self._get_val_from_obj(obj)
    return value.raw

Missing import to settings

http://github.com/jamesturk/django-markupfield/blob/master/markupfield/markup.py#L69

There is a reference to settings in the above line but no import. As reported by my colleague Javed Khan.

Verbose_name to "Wording markup type"

Well, I try to change a verbose_name of this field, but didn't success. How can do that?

Incrementing `creation_counter` breaks field equality

These lines cannot be used to preserve ordering in forms, because Django's Field.__eq__ uses the creation_counter for equality:

markup_type_field.creation_counter = self.creation_counter + 1
rendered_field.creation_counter = self.creation_counter + 2

Unless the MarkupField is the last field in a model class, this causes a collision of creation_counter with the fields that come after. This breaks the Field equality check and hence any code that relies on field equality. For instance, I came across this bug when using django-concurrency, which relies on field equality to detect the version field.

django-markupfield does not currently support Django 1.7 migrations

When migrating a model with Django 1.7b3, I end up getting the error message:

django.db.utils.OperationalError: duplicate column name: desc_markup_type

models.py

class Project(models.Model):
    name = models.CharField(max_length=80)
    desc = MarkupField(escape_html = True, null=True)

0001_initial.py

class Migration(migrations.Migration):

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name=b'Project',
            fields=[
                ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
                (b'name', models.CharField(max_length=80)),
                (b'desc', markupfield.fields.MarkupField(null=True)),
                (b'desc_markup_type', models.CharField(default=None, max_length=30, choices=[(b'', b'--'), (b'html', b'html'), (b'plain', b'plain')])),
                (b'_desc_rendered', models.TextField(editable=False)),
            ],
            options={
            },
            bases=(models.Model,),
        ),
    ]

Quick-fix: You can temporarily fix this issue by commenting you b'desc_markup_type and _desc_rendered in the Migration class. Updating a model that way won't work, though (unless you don't mind removing the whole MarkupField temporally).

:linenos: param is not parsed??

Hi,

I have following code-block

.. code-block:: HTML
   :linenos:

Then it is giving error like Error in "code-block" directive: unknown option: "linenos".

What is the issue? I am using markfield 1.4.0, and docutils 0.12. My setting is like:

import markdown
from docutils.core import publish_parts

def render_rest(markup):
    parts = publish_parts(source=markup, writer_name="html4css1")
    return parts["fragment"]

MARKUP_FIELD_TYPES = (
    ('markdown', markdown.markdown),
    ('ReST', render_rest),
)

Thanks, and I select markup type ReST while saving.

Preventive maintenance of package

At the moment, it is unclear how this package is being maintained. Everything still seems to work like it should, but there currently aren't any efforts for preventive maintenance to make sure that it stays that way. For example the current tox setup only runs tests for versions of Django that aren't supported anymore.

I'd be happy to help and send a pull request to fix this, and would also like to help in any other way that I can. Can you point out any other parts that I probably also need to take a look at?

Push a new version to Pypi

I just noticed that the version in pypi lack of some fixes. Could you please push a new version? Thanks.

this project could use a new maintainer

I haven't used django-markupfield in 3-4 years, though I continue to make minor bugfixes/etc. If someone else is using it and would like to take ownership I'd happily transfer ownership over to you if you get in touch and explain what you'd like to do/how you use django-markupfield.

Settings?

I'm somewhat unclear on where the settings belong.

Do the markup type choices belong in the model file of each model? Or do they go into myapp/settings.py?

Thanks for create this django extension, it looks like it might be exactly what I wished for.

Table didnt appear !!

MarkupDescriptor breaks with Django 2.1's admin checks

Problem

Django 2.1's system checks no longer recognises markup fields as existing when an admin's list_display refers to them:

<class 'ExampleAdmin'>: (admin.E108) The value of 'list_display[5]' refers to 'example_markup_field', which is not a callable, an attribute of 'ExampleAdmin', or an attribute or method on 'ExampleModel'.

(This works fine in Django 2.0.)

Analysis

This seems to be due to the following change introduced in Django 2.1:

• Ticket #28490: Descriptors on Models are reported as nonexistent by System Check Framework for ModelAdmin.list_display if they return None
  
• PR: django/django#8897
• (Commit django/django@47016ad)

django-markupfield's MarkupDescriptor raises an AttributeError when accessed via the model class. In Django 2.0, this was still handled in the else clause of _check_list_display_item, but in Django 2.1, this is no longer the case, and it interprets the AttributeError as the field being missing.

Fix

The convention of other Django field descriptors' __get__ implementations when accessed via the class seems to be to return the descriptor instance itself:

        if instance is None:
            return self

Changing MarkupDescriptor to follow this same convention should fix this.

markdown should be added as a dependency

Hello,

in your guide is that this should be added to settings.py

import markdown
from docutils.core import publish_parts

def render_rest(markup):
    parts = publish_parts(source=markup, writer_name="html4css1")
    return parts["fragment"]

MARKUP_FIELD_TYPES = (
    ('markdown', markdown.markdown),
    ('ReST', render_rest),
)

• you import markdown there, hence, it should be installed with pip as a dependency.

xss vulnerability

django-markupfield is vulnerable to xss since it marks the input as safe. The attached patch fixes this by escaping input before giving it to the markup renderers.
http://gist.github.com/522548

ArrayField(MarkupField()) support

I think that ArrayFields of MarkupFields doesn't work

non-lazy gettext calls at import time

Django==1.7.7

For example, ugettext() uses the app registry to look up translation catalogs in applications. To translate at import time, you need ugettext_lazy() instead. (Using ugettext() would be a bug, because the translation would happen at import time, rather than at each request depending on the active language.)

Render inside form-tools formwizard

I'm using form-tools and I cannot seem to get the Markdown Field to show up on the page:

{% block formpart %}
    {{ form.errors }}
    <span class="title" style="font-size: 1.5em;">Name of <span class="highlight">The Project</span></span>
    <div class="formElems">
        {{ form.name|attr:"class:descript"|attr:"rows:5"|attr:"placeholder:Project Name" }}
    </div>
    <span class="title" style="font-size: 1.5em;">What are your <span class="highlight">Project Objectives?</span></span>
    <div class="formElems">
        {{ form.objective.rendered }}
    </div>
    <span class="title" style="font-size: 1.5em;">What is the <span class="highlight">Background</span> for your project?</span>
    <div class="formElems">
        {{ form.background|attr:"class:descript"|attr:"rows:5"|attr:"placeholder:Project Background" }}
    </div>
{% endblock %}

Markdown footnote reset issue

This is probably a documentation issue more than anything else. When configuring Markdown, the first instinct is probably to use a configured markdown.Markdown instance in MARKUP_FIELD_TYPES. However, Markdown instances need to be explicitly reset (Markdown.reset()) after rendering to avoid issues such as this Python-Markdown/markdown#401 (I tripped on this, so here I am).

markdown.markdown handles this automatically, so the proper way, probably, to go about this is to use django.utils.functional.curry (or own similar function) to wrap it (like you do when you're adding Pygments extension) and add configuration in MARKUP_FIELD_TYPES. This should probably be mentioned in documentation.

Django Modelform not saving field (Django 1.9)

I just ran into an issue saving a markupfield from django admin. But the field isn't updated instead, it escapes the existing rendered value. Saving the field manually works but not when using django model forms.

disallow html

Is it safe to put this in my settings.py?

MARKUP_FIELD_TYPES = tuple( i for i in markup.DEFAULT_MARKUP_TYPES if i[0].lower() != 'html' )

Markdown is stored as HTML in the database

I have a field declared as: MarkupField(blank=True, markup_type='markdown'). When I save this field in the database, it's converted to HTML. How do I make it store as markdown in the database?