Thursday, January 28, 2010

Django and Ajax Form Submissions

I'm writing this post, in part, to correct a mistake. In Chapter 10 of the book, I cover how to add Ajax functionality to our Django e-commerce site by using the jQuery library. In the interest of keeping the chapter short and easy to understand, I kept the coverage in that chapter very rudimentary.

In hindsight, it was, I believe, a little too rudimentary, and I don't think the code is nearly as good as it could be. It's not very DRY (since we repeat ourselves), it's not degradable (if the user has JavaScript turned off, it doesn't work), and for larger forms with lots dozens of fields, it's likely to become a maintenance headache and possibly hurt performance.

This isn't errata, but it's worth taking another look at. Let's revisit how to do a simple Ajax form submission in a Django project with jQuery, using the submission of product reviews as an example.

(I'm assuming, if you're reading this, that you have the book handy and can follow along with what I'm doing. If you don't, I'll try to explain the concepts well enough so that you don't need the book. Just keep in mind there's a ProductReview model and associated form class being referenced here that I haven't included.)

Let's start with the URL entry that links to the view that accepts product review data, on page 219:

urlpatterns = patterns('ecomstore.catalog.views',
  ( r'^review/product/add/$', 'add_review' ),
)

One of the main problems is that this URL is defined here, and before, in our JavaScript code, we defined the same URL again, in perfect violation of the DRY principle. We're going to fix this. The first step is to give this URL entry a name parameter:

urlpatterns = patterns('ecomstore.catalog.views',
  ( r'^review/product/add/$', 'add_review', {}, 'product_add_review' ),
)

That means that, in our templates, we can use the URL tag and refer to 'product_add_review' to reference this URL. If it's in the template, we can make sure it's in the DOM someplace, and if it's in the DOM, then we can make our JavaScript code aware of it. That's exactly what we're going to do next.

In the book, on page 214, the fields are just text inputs on the page, and aren't actually contained in a form element. Actually, they're in a div element with an id attribute of 'review_form'. This is, at best, a missed opportunity.

Here's the new template code. Again, if you don't have the book, just know that 'review_form' is an instance of the review form class and that 'p' refers to the product instance on the current product page:

<div id="review_form">
<form id="review" action="{% url product_add_review %}" method="post">
  <div id="review_errors"></div>
  <table>
    {{ review_form.as_table }}
    <tbody><tr><td colspan="2">
      <input id="id_slug" value="{{ p.slug }}" type="hidden">
      <input id="submit_review" value="Submit" type="submit">
      <a href="javascript:void(0);" id="cancel_review">Cancel</a>
    </td></tr>
  </tbody></table>
</form>
</div>

With this, the DOM has all the semantic information it needs for posting a review form, and it can pass that along to our JavaScript. Also notice that I changed the "Cancel" button from a submit input to an anchor element.

Now, the changes we're going to make to our JavaScript are more than just a little superficial. If you're sitting there with the book in front of you, you won't be able to just change the addProductReview() function and have it work, because we're going to change the events that are fired and how we're attaching those events.

First, we're going to attach the addProductReview() to the form element. Inside of the prepareDocument() function, the contents of which are listed in the middle of page 216, remove this line:

jQuery("#submit_review).click(addProductReview);

and replace it with this:

jQuery("form#review").submit(function(e){
  addProductReview(e);
});

Now, the addProductReview() function will fire whenever someone submits the form, which is done by click the "Submit" button.

Let's turn our attention to the addProductReview() function itself and give it an extreme makeover. In the book, we use the jQuery.post() function to submit our Ajax request, but here, I'm going to use the jQuery.ajax() function, which allows for some more fine-grained control over the submission process. Here is the reworked version, which I will explain, line-by-line, in a moment:

function addProductReview(e){
  e.preventDefault();
  var review_form = jQuery(e.target);
  jQuery.ajax({
    url: review_form.attr('action'),
    type: review_form.attr('method'),
    data: review_form.serialize(),
    dataType: 'json',
    success: function(json){
      // code to update DOM here
    },
    error: function(xhr, ajaxOptions, thrownError){
      // log ajax errors?
    }
  });
};

This new addProductReview() function takes a single argument, 'e', which refers to the form submission event itself, passed down on high from when we attached the function to the submit form event. 'e' allows us to do a couple of interesting things.

First, we call preventDefault() on the event. That means that the JavaScript will halt the normal form submission, so the browser won't reload the page. Naturally, if the user has JavaScript disabled in their browser, e.preventDefault won't fire, and the form will post to the "Add Review" URL as usual, the non-ajaxy way. This is the first step to ensure that our form degrades gracefully. (We'll revisit this is a moment, when we look at the view function.)

Next, we can reference the form element itself by using the selector and e.target:

var review_form = jQuery(e.target);

'review_form' now refers to the 'form#review' element on the page, which includes all of its attributes and child elements, including the form inputs themselves. We're going to use this for the values we submit with our Ajax request.

The jQuery.ajax() function takes a few parameters. Here are the ones that I've used, along with a quick definition of each:

1. url - The URL path to which we should submit the request.
2. type - the HTTP method, either "get" or "post".
3. data - the form values, as a set of name-value pairs encoded in URL format. (e.g. "name=john&amp;content=good book!")
4. dataType - the type of data we expect as a response. (in this case, "json")
5. success - a function that handles the response after a successful request.
6. error - a function to handle unsuccessful requests. (optional)

The first three items, we get right from the form contents itself. The first two, we obtain by using the attr() function to get the values of the attributes of the form element:

url: review_form.attr('action'),
type: review_form.attr('method'),

The third one uses a new function that allows us to get the values of the form inputs as a set of name-value pairs, as a single URL string:

data: review_form.serialize()

That's much easier than having to spell out a selector and create a new variable for each input on the form, as was done in the book. This is especially true if you have a form with dozens of fields. The serialize() function is a handy shortcut. Also, DOM selection is an expensive operation for JavaScript, so doing a single selection of the form element and serializing it is much quicker than selecting the elements one-by-one, from a performance perspective.

For the last few items: we expect the dataType the server returns to be a JSON object. Inside of the success function, you can define the same code that was in the book, since the DOM update operations will be largely the same. Lastly, while we didn't create an architecture to handle Ajax errors in the book, I included the block where you would put code to handle any errors your Ajax request encounters. This could be useful for logging purposes.

Now that this is done, we just need to make a couple of small changes to our view function. First of all, requests that come into this URL from form submissions might be Ajax requests, or they could be coming from users who have JavaScript turned off and have just submitted a form the traditional, non-Ajax way. Our view function needs to check this and respond accordingly.

Here is the new view function in catalog/views.py:

@login_required
def add_review(request):
if request.method == 'POST':
  form = ProductReviewForm(request.POST)
  slug = request.POST.get('slug')
  product = Product.active.get(slug=slug)

  if form.is_valid():
    review = form.save(commit=False)
    review.user = request.user
    review.product = product
    review.save()

    template = "catalog/product_review.html"
    html = render_to_string(template, {'review': review })
    response = simplejson.dumps({'success': 'True', 'html': html})

  else:
    html = form.errors.as_ul()
    response = simplejson.dumps({'success':'False', 'html':html})

  if request.is_ajax():
    return HttpResponse(response,
      content_type="application/javascript")
  else:
    return HttpResponseRedirect(product.get_absolute_url())

Again, I haven't defined the Review model or form in this case, but you should get the idea. We still handle the validation and saving of the new review instance in the same fashion. Towards the end, we just conditionalize the type of response we return depending on the nature of the request. If it's an Ajax request, we return the JSON object to the JavaScript function that we assume called it. Otherwise, it's a non-Ajax request coming from someone who has JavaScript disabled, and we just reload the current product page, with their new review posted.

They key here is in using request.is_ajax() to check the nature of the request. jQuery, as well as most other major JavaScript libraries, adds a header to the request called HTTP_X_REQUESTED_WITH with a value of "XMLHttpRequest". If Django finds this header in the request with that value, request.is_ajax() returns True.

There's one small problem with this: if a user has JavaScript turned off, and they submit a product review that doesn't validate, we don't communicate this to the user. They just get the page reloaded without their product review and no indication of what went wrong. In the case of our site, we already display a warning to people with JavaScript disabled that stuff might not work as expected. (Depending on your requirements, that might be unacceptable, but I leave it up to you, and the architecture of your own site, to determine how you could fix that particular bug.)

To recap, we've made the following improvements, which I encourage you to do elsewhere in your Django projects:

1. Give the Ajax request URL a name in your urls.py file, and use the {% url %} tag in the DOM of your page so jQuery can access it. (If the action attribute should be the current path, you can use {{ request.path }} in most cases in your templates to spell that out explicitly.)

2. Structure the elements of your page so they function all right even if the user has JavaScript disabled in their browser.

3. For forms, define the method for submission in the 'method' attribute of the form and use jQuery's attr() function to get it.

4. You can serialize the contents of a form for Ajax requests using the serialize() function on the form element.

5. In the view function, use the request.is_ajax() method to determine if the request came in via an Ajax request and, based on the origin, send the appropriate response.

As a side note, a lot of readers have complained about problems with the djangodblog app. For one, the installation instructions in the book don't work with current versions, but more importantly, it seems that if your database uses the UTF-8 character set encoding, django-db-log is not compatible with it. That's a bit disappointing, and I'm trying to figure out what to do about it.

9 comments:

  1. Great post! very helpful indeed :D

    I was wondering if you know any way to do DJango form validation within an ajax call to a view.

    I would like to do something like this:
    if request.is_ajax():
    form.is_valid()

    And if there are errors return them so JQuery can parse and show them :D

    Thanks

    ReplyDelete
  2. Hi Miguel,

    I do that quite often, and it's similar to the way you described. The way I like to do it is to return the form errors rendered as a UL element or return the bound form rendered as HTML with the errors.

    if request.is_ajax() and request.method == 'POST':
    form = MyForm(request.POST)
    if form.is_valid():
    pass # do something here - redirect(?)
    else:
    form_html = form.as_table()
    data = {'form_html': form_html}
    json = simplejson.dumps(data)
    return HttpResponse(json)

    Then, have jQuery update the DOM with the newly bound form data, which should contain the errors.

    Optionally, you might work this flow into the normal view function for the page, so that if the user has JavaScript disabled, the form submissions will still process correctly and render errors to the user.

    ReplyDelete
  3. Hello Jim,

    I am following you in the book and I would like to point to two typos I found in this post, so others will not spend hours finding what's wrong...

    First is in the view function.
    This line
    slug = request.POST.get('slug')
    should be like this
    slug = request.POST.get('id_slug')

    Second is in the function in script.js file.
    This line
    success: function(json){
    should be like this
    success: function(response){

    It took me a while to find what is going on here, I am very newbie......

    Anyway, thank you very much for a great tutorial!

    ReplyDelete
    Replies
    1. I replaced this line:
      slug = request.POST.get('slug')
      with
      slug = request.POST.get('id_slug')
      in views.py however the slug is always None! If I set the slug explicitly to an available product slug, everything that follows works perfectly! Why the 'id_slug' is not passed in the request.POST.. I made sure it is written correctly in the template! thx a lot!

      Delete
  4. What version of Django does this require? I'm using 1.0.

    ReplyDelete
  5. HI DF,

    This should work with Django 1.0 without any problem. The example in the book uses 1.1, but I don't see anything here that changed. For Django 1.2 and later, it might be necessary to add the {% csrf_token %} tag to the form to avoid the HTTP 403 Forbidden security warning, which is more likely to come up since the CSRF protection is enabled by default.

    ReplyDelete
  6. First i'd like to thank you for the best tutrial on the net about Django .

    Beacause i am using the version 1.4a1 of django, i just want to know how to handle CSRF protection on jquery/ajax used in posting previews on products(mentioned on page 217).

    in other words i tried to modify the function addProductReview() in the file scripts.js as it is said in this tutorial:
    https://docs.djangoproject.com/en/dev/ref/contrib/csrf/#how-to-use-it

    but i could not how to do it

    thanks in advance

    ReplyDelete
    Replies
    1. Hi redatest-

      There's not a lot of detail to go on here. So you added the {% csrf_token %} inside of the form element, but you're still getting an error? What kind of error are you seeing?

      Feel free to shoot me an email to get this sorted out: jim@django-ecommerce.com. Thanks!

      Jim

      Delete
  7. More details were sent in a personal message.

    I'll post thr solution if we solve the problem

    ReplyDelete