A basic search box with Django
29 November 2009
For providing a simple search box for this blog, I was using a search engine's custom search facility. This approach presented a number of shortcomings.
Firstly, new posts were not represented in the search results as posts only go into the search engine when the search engine bothers to send its indexing scripts at the site.
Secondly, the search engines do not distinguish between the content of the post and the general template of the site.
Thirdly, the results page from the search engine was cluttered and rather unhelpful. What I really wanted was a list of posts that have the result in, not the same post several times in different versions (normal view, category view, RSS feed, etc).
The shortcomings go on and on, suffice to say that I thought I should really write a proper search form. This post explains how I did it.
This site has been written using the Django web framework, I open sourced the code into the Soturi project. All the posts are held as a simple text field in an SQL database. Using a search engine library like Lucene is somewhat overkill. We can just use something like:
from blog.models import Post
Post.objects.filter(body__contains='Django')
This gives the posts containing the word Django, ordered by the default ordering, which is last post date. We can of course order the results by anything we like, but this order is probably what we want anyway.
Website searches tend be quite inexact, so we should at least support case insensitive searches, i.e. so that 'django' will match 'Django'. So we use the icontains lookup which provides case-insensitive results:
from blog.models import Post
Post.objects.filter(body__icontains='django')
However, we don't want to just be able to search by one keyword alone, we want to be able to search by lots of keywords. We also want to be able to treat "something in double quotes" as a single keyword.
So the first thing we need is a function that splits the query into keywords:
def split_query_into_keywords(query):
"""Split the query into keywords,
where keywords are double quoted together,
use as one keyword."""
keywords = []
# Deal with quoted keywords
while '"' in query:
first_quote = query.find('"')
second_quote = query.find('"', first_quote + 1)
quoted_keywords = query[first_quote:second_quote + 1]
keywords.append(quoted_keywords.strip('"'))
query = query.replace(quoted_keywords, ' ')
# Split the rest by spaces
keywords.extend(query.split())
return keywords
MYQUERY = """Django form "aggregated values" """
split_query_into_keywords(MYQUERY)
Django queries are lazy, i.e. they do not actually run until the results are used which then forces the query to be evaluated. Therefore we can just chain filters together and Django should only access the database once.
from blog.models import Post
def search_for_keywords(keywords):
"""Make a search that contains all of the keywords."""
posts = Post.objects.all()
for keyword in keywords:
posts = posts.filter(body__icontains=keyword)
return posts
MYQUERY = """Django form "aggregated values" """
keywords = split_query_into_keywords(MYQUERY)
posts = search_for_keywords(keywords)
One could of course also search the other fields such as title, author and comments, then combine the results together. However, this is good enough for a simple search.
The rest of the search is just a basic Django form, as explained in the Django forms documentation. I.e. we have the form class:
from django import forms
class SearchForm(forms.Form):
"""Search posts by keywords"""
keywords = forms.CharField(max_length=100)
For processing the form, I wrote something like this in the view:
if request.method == 'POST':
form = SearchForm(request.POST)
if form.is_valid():
keywords = form.cleaned_data['keywords']
keyword_list = split_query_into_keywords(keywords)
posts = search_for_keywords(keyword_list)
if posts:
# Show the results
Lastly I put the HTML form tags in the template, and a line into urls.py and that was about it.
You could create a new template for the search results, I personally didn't bother, I just reused the list of posts which already existed.
Merry coding!
