Using custom model_to_dict() in Django
2016-12-09
In Django, when converting a database model to a dictionary, I usually used to convert only the member variables corresponding to the column of the table to handle the response to the request in the form of Json and so on. But, whenever you encounter a member variable of type ManyToManyField, it would be too cumbersome to write three or four lines of code each time to loop around to get their values from the member variables of the ManyToManyField in a model object.
For example, suppose you have a modeling class of Publication, Article as follows.
from django.db import models
class Publication(models.Model):
title = models.CharField(max_length=30)
def __str__(self):
return self.title
class Meta:
ordering = ('title',)
class Article(models.Model):
headline = models.CharField(max_length=100)
publications = models.ManyToManyField(Publication)
def __str__(self):
return self.headline
class Meta:
ordering = ('headline',)
For the Publication instance, you can convert it to a dictionary by calling the model_to_dict() function only once. However, when converting an instance of Article with a member variable of ManyToManyField to a dictionary, you have to invoke model_to_dict() or loop around more than once to get their values because ManyToManyField is a list type.
Whenever I encountered ManyToManyField, I used to create a separate method, or loop it, and convert it each by each. Suddenly, every time I write a for statement or a separate function, I thought it would be better to solve it all once.
For example, let’s say you have a property in the Article class, a member function, and several ManyToManyFields like this:
class Article(models.Model):
headline = models.CharField(max_length=100)
publications = models.ManyToManyField(Publication)
tags = models.ManyToManyField(Tag)
def convert_cover_img(self):
...
@property
def count(self):
...
def __str__(self):
return self.headline
class Meta:
ordering = ('headline',)
What I want is to specify each values to be transformed, it could be such as member variables, member functions, and properties of the modeling class, as shown below, and the whole instance is converted into a single dictationary chunk in a single call. For example, if you need only one of the member variables of ManyToManyField, you can convert it to __[field_name] by specifying the desired field name as follows.
FIELDS = {'id', 'headline', 'publications__title', 'tags__name', 'convert_cover_img', 'count']
result = OrderedDict()
result['data'] = [model_to_dict(instance=obj, fields=FIELDS) for obj in page.object_list]
result['has_next'] = page.has_next()
...
return JsonResponse(format_response(result, RES_OK))
Unfortunately, if you need more than one column for ManyToManyField to build the response like json form, you have no choice but to define a seperate property or function in a model class. In a specific case, for example, if you are extending all attributes - like id, name and etc - of the tags member variable ManyToManyField while converting the Article instance.
@property
def tags_to_dict(self):
...
However, since the model class describes the process of transforming their member variables, the above manner has the advantage that it is well modularized and reusable. As a result, the entire code is really simple and neat, because it can hide the process of converting an instance to a dictionary.
Now we no longer have to loop around to handle ManyToManyFields or write a separate function. It also eliminates the need to worry about member properties or member functions, which is not modeling fields .
Here is the code for model_to_dict() that was created. Once the conversion code of the a model is unified into just one line as shown below, repetitive and annoying coding for conversion is no longer needed. Since the parent class is created and the most job in the inherited child class is just to specify only the fields to be converted. As a result, the development speed can be much faster, which can improve the quality of the query set manager or focus on more creative work.
def model_to_dict(instance, fields=None, exclude=None, date_to_strf=None):
from django.db.models.fields.related import ManyToManyField
from django.db.models.fields import DateTimeField
from django.db.models.fields.files import ImageField, FileField
opts = instance._meta
data = {}
"""
Why is `__fields` in here?
it holds the list of fields except for the one ends with a suffix '__[field_name]'.
When converting a model object to a dictionary using this method,
You can use a suffix to point to the field of ManyToManyField in the model instance.
The suffix ends with '__[field_name]' like 'publications__name'
"""
__fields = list(map(lambda a: a.split('__')[0], fields or []))
for f in chain(opts.concrete_fields, opts.virtual_fields, opts.many_to_many):
is_edtiable = getattr(f, 'editable', False)
if fields and f.name not in __fields:
continue
if exclude and f.name in exclude:
continue
if isinstance(f, ManyToManyField):
if instance.pk is None:
data[f.name] = []
else:
qs = f.value_from_object(instance)
if qs._result_cache is not None:
data[f.name] = [item.pk for item in qs]
else:
try:
m2m_field = list(filter(lambda a: f.name in a and a.find('__') != -1, fields))[0]
key = m2m_field[len(f.name) + 2:]
data[f.name] = list(qs.values_list(key, flat=True))
except IndexError:
data[f.name] = list(qs.values_list('pk', flat=True))
elif isinstance(f, DateTimeField):
date = f.value_from_object(instance)
data[f.name] = date_to_strf(date) if date_to_strf else date_to_timestamp(date)
elif isinstance(f, ImageField):
image = f.value_from_object(instance)
data[f.name] = image.url if image else None
elif isinstance(f, FileField):
file = f.value_from_object(instance)
data[f.name] = file.url if file else None
elif is_edtiable:
data[f.name] = f.value_from_object(instance)
"""
Just call an instance's function or property from a string with the function name in `__fields` arguments.
"""
funcs = set(__fields) - set(list(data.keys()))
for func in funcs:
obj = getattr(instance, func)
if inspect.ismethod(obj):
data[func] = obj()
else:
data[func] = obj
return data
You might have performance problems, but if you have a lot of cost to convert an instance, you might be able to run 10 simultaneously using a coroutine, etc. Of course, if the order is important, when you collect the results, there may be a hassle, such as having to rearrange them in order.