JSON API authentication for Rails application with devise

In this post I will discuss how I set up JSON API authentication with devise gem for Rails application with React on the frontend. I will show my initial setup and the final setup and will talk about reasons why I decided to refactor and set the Rails routes the way I did.

If you're in a hurry and just need a step by step implementation guide, there is a Steps to follow section and a link to the demo application.

OUTLINE

1. Why?
2. Introduction to devise
3. Setting up devise
4. Authentication. Solution #1
5. Authentication. Solution #2
6. Steps to follow
7. Conclusion

---

Why?

A couple of months ago, I decided that I wanted to improve my skills outside of work. After discussing it with several people, I decided that there's no better way to improve than start a personal project.

I work mainly on full-stack projects (React + Rails), and in our team, we don't divide tasks into frontend or backend. So I went with a similar tech stack and bootstrapped a Rails application with React.

The application itself is pretty straightforward - it suggests a lunch place. As a user, you can sign in, add/remove lunch places from a list, and click a suggestion button that randomly indicates a lunch place. Nothing complicated here, but there is still a long way to go.

Introduction to devise gem

As I mentioned before, my application has a User who can sign up/sign in/sign out. So, for me, it meant that the first step was to add user authentication.

I haven't implemented authentication from scratch in a veeeery long time. I had only basic knowledge about the whole authentication process, and I was looking for a gem that would help me with authentication.

A gem called devise seemed to be a popular choice in the community. There are several tutorials on how to set it up with Rails apps. There is also a great documentation which is easy to follow, even for a recent junior like me.

Devise follows the Model-View-Controller pattern (same as Rails) and has helper methods to authenticate a user or check a current authenticated user.

It also comes with different configuration options for authentication model. Some of them are added to the model by default after adding the gem, e.g. Rememberable which generates and verifies a secret token. Options can be added later, e.g. Confirmable which sends the confirmation email to the user.

As for the authentication flow, I had to spend some time to understand what's going on. From all the information found on the internet + discussions with other developers, this is what I believe a default devise authentication flow looks like 👇

1. Devise creates a secret token
2. Stores it in the Rails session store (or Redis or any other session store, depends on your application's configuration)
3. Secret token is sent to the browser via Session Cookie (Set-Cookie header) on successful sign-up/sign-in
4. Browser saves session cookie and adds automatically with each request to the server
5. Browser will remove session cookie when it’s expired or when user signs-out

According to the docs, it's also possible to use JWT tokens instead of secret token, but I haven't explored that option yet, so be my guest 😆

Setting up authentication with devise

To figure out what I would need to refactor to make devise work with my application, I wanted to see how to set up devise and what it generates. I created a demo application using the same rails new command I used to bootstrap my lunch application, add User model and then installed devise gem.

✏️ Note:
I was getting the following error when adding devise for User model that already had an email column. If you have to have a model that you're planning to add authentication to, make sure to initialize it without email column.

ActiveRecord::StatementInvalid: PG::DuplicateColumn: ERROR:  column "email" of relation "users" already exists

As documentation suggested, I ran rails generate devise user, which was suppose to

...configures your config/routes.rb file to point to the Devise controller.

Immediately after running this command rails console displayed a following message informing that it added a new route.

After that I ran the migration files with with rails db:migrate command. Checked my routes.rb file and, indeed, it now included devise_for :users.

Rails.application.routes.draw do
    devise_for :users
end

Sent a request to the /users/sign_in endpoint, and as expected, it was rendering a static view 🎉

---

Authentication. Solution #1

Since I used Rails as a server and React on the frontend, which was supposed to send requests and be responsible for the views, I had to make sure to:

1. nest devise routes under /api/v1 to differ from future static routes
2. make devise controllers accept and return JSON

It was pretty easy to achieve the first, as the devise documentation clearly stated that

devise_for is meant to play nicely with other routes methods. For example, by calling devise_for inside a namespace, it automatically nests your devise controllers: source

I refactored Rails routes to 👇

Rails.application.routes.draw do
  namespace :api do
    namespace :v1 do
      devise_for :users
    end
  end
end

Ran rails routes --expanded again, and I saw this result.

--[ Route 1 ]----------------------------------------------------------------------------------------------------------------------------------
Prefix            | new_api_v1_user_session
Verb              | GET
URI               | /api/v1/users/sign_in(.:format)
Controller#Action | api/v1/sessions#new
--[ Route 3 ]----------------------------------------------------------------------------------------------------------------------------------
Prefix            | destroy_api_v1_user_session
Verb              | DELETE
URI               | /api/v1/users/sign_out(.:format)
Controller#Action | api/v1/sessions#destroy
--[ Route 10 ]---------------------------------------------------------------------------------------------------------------------------------
Prefix            | new_api_v1_user_registration
Verb              | GET
URI               | /api/v1/users/sign_up(.:format)
Controller#Action | api/v1/registrations#new

Good, but all this did was just nesting the routes under /api/v1. It was still necessary to figure out how to make devise handle JSON. Checking the routes right after adding devise authentication to User model helped to identify controllers I needed to override to make devise work with JSON.

• SessionsController - responsible for sign_in/sign_out
• RegistrationsController - responsible for sign_up

I found a solution on how to override devise controllers in the official documentation. For the JSON part there is this awesome article. TLDR, to make devise accept and respond to JSON it's necessary to override those two controllers by adding a respond_to :json line to both.

Your controllers should look like this 👇

✏️ Note:
I had to skip CSRF authentication completely in the demo app to make routes work. Please keep in mind that this is a demo application, and in reality, you should not skip CSRF token validation.

✏️ Note:
It's not necessary to put controllers into /api/v1 folder, I did it because I wrapped my routes into :api, :v1 namespace Just make sure that namespace in routes.rb file is same as the path to your custom controllers.

RegistrationsController

responsible for sign up

# frozen_string_literal: true

module Api
  module V1
    class RegistrationsController < Devise::RegistrationsController
      respond_to :json

      skip_before_action :verify_authenticity_token
    end
  end
end

SessionsController

responsible for sign in/sign out

# frozen_string_literal: true

module Api
  module V1
    class SessionsController < Devise::SessionsController
      respond_to :json

      skip_before_action :verify_authenticity_token
    end
  end
end

According to the documentation's Configuring routes section, passing a controllers argument to devise_for in routes.rb file would make devise use my custom controllers and be within the /api/v1 scope.

I refactored my routes.rb file like so 👇

Rails.application.routes.draw do

  namespace :api do
    namespace :v1 do
      devise_for :users, controllers: { sessions: 'api/v1/sessions', registrations: 'api/v1/registrations' }
    end
  end
end

and I tried to sign up a new user and sent the following request 👇

URL: /api/v1/users
Method: POST
Body (JSON): {
    user: {
        email: test@mail.com,
        password: 123456,
        password_confirmation: 123456
    }
}

Everything seemed to be correct according to the documentation...
...buuuuuut it didn't work 😑

The request resulted in a 422 Unprocessable Entity error stating that email and password can't be blank. It seemed like devise wasn't receiving user params.

{
  "errors": {
    "email": [
      "can't be blank"
    ],
    "password": [
      "can't be blank"
    ]
  }
}

I re-read the documentation (banging my head against the wall a tiny bit) and found an alternative way to tell devise that it should use my customer controllers. Apparently, you can also wrap custom routes and specify custom controllers with a devise_scope block.

I refactored the routes again

Rails.application.routes.draw do
  devise_for :users

  namespace :api do
    namespace :v1 do
      devise_scope :user do
        post 'sign_up', to: 'registrations#create'
        post 'sign_in', to: 'sessions#create'
      end
    end
  end
end

✏️ Note:
It is still necessary to keep the devise_for :users in the routes.rb file to use helper methods, e.g., current_user.

I sent the same request to sign up a new user, but this time I was using the custom api/v1/sign_up endpoint which I declared earlier.

URL: /api/v1/sign_up
Method: POST
Body (JSON): {
    user: {
        email: test@mail.com,
        password: 123456,
        password_confirmation: 123456
    }
}

...aaaand it worked!! 🎉

I created a new user by sending a JSON request (which my React frontend is suppose to do) and received a JSON response.
Please ignore the username: null field in the response, I didn't add any validation since it was a demo app 🙈

✏️ Note:
If you're also using Insomnia to test APIs (not sure about other API clients), don't forget about Accept: application/json header to see the JSON response.

---

Authentication. Solution #2

I was happy and dancing and partying for about 20 minutes straight 💃 And then I decided to test the sign out endpoint...

I checked my current routes with rails routes --expanded and found the one I needed

--[ Route 3 ]-------------------------------------------------------------------------------------------------------------
Prefix            | destroy_user_session
Verb              | DELETE
URI               | /users/sign_out(.:format)
Controller#Action | devise/sessions#destroy

Sent a request

URL: /users/sign_out
Method: DELETE
Body (JSON): {
    user: {
        email: test@mail.com
    }
}

and I received a 422 Unprocessable Entity error with ActionController::InvalidAuthenticityToken❗

But wait a moment, I already skipped CSRF token check for both Registrations and Sessions controllers. So how was I still getting the CSRF validation error?

It was because I made a wrong assumption ❗
I assumed that the paths I didn't mention under devise_scope :user would fall back to the default route names, but they will still use my custom controllers which had the CSRF token verification disabled. So this is when it hit me that with this setup I would need to overwrite ALL the endpoints.

If I was a little bit more careful, I would see that the Controller#Action of the sign_out route was handled by devise/sessions#destroy action! It might have been straightforward for some people from the beginning, but I was too busy being happy about the fact that authentication has finally worked 😆

Of course, I could add the remaining endpoints to make it work, but I didn't want to rewrite all the endpoints since I didn't have any custom code to handle authentication. After all, I just needed to add the respond_to :json line to the controllers.

I decided to research a bit more because I thought that there had to be a better way to have the /api/v1 prefix and use custom controllers 🤔
After some time, I found a reference in the rubydoc devise documentation that it was also possible to use the path option which

allows you to set up path name that will be used, as rails routes does. The following route configuration would set up your route as /accounts instead of /users: devise_for :users, path: 'accounts'

This seemed like something I needed to make the initial configuration work because the only reason I didn't end up using devise_for was the error related to params. So, I decided to try again, and instead of nesting devise_scope under the namespace :api, :v1, I used devise_for and put it at the top of the file while passing the /api/v1 as the path option.

Rails.application.routes.draw do
  devise_for :users, path: 'api/v1/users', controllers:
    {
      sessions: 'api/v1/sessions',
      registrations: 'api/v1/registrations'
    }
end

It worked like a charm 🎉

---

Steps to follow

This assumes that you already have a Rails application with a User model that requires JSON authentication.

First add devise gem to your Gemfile and

1. run bundle install
2. run rails generate devise:install - After this, you will see a list of instructions. Make sure to follow the first one that is marked with Required for all applications.
3. run rails generate devise MODEL_NAME - in my case it was rails generate devise user
4. run rails db:migrate
5. run rails routes --expanded and check the routes; they should include endpoints for sign_up, sign_in, and sign_out

Next we're going to override default devise routes and make them use our custom controllers to enable accepting and responding with JSON.

1. create SessionsController, RegistrationsController and add respond_to :json line to both of them
2. don't forget to put them into correct folder, e.g. in my case they were in /controllers/api/v1 folder

As for the routes.rb file. You can either use devise_scope and configure all paths manually (details in Authentication Solution #1)

OR you can use devise_for and specify only two controllers and set the api/v1 with path option (details in Authentication Solution #2)

And as promised, here is a link to a working application.

---

Conclusion

TBH, I still don't have a clue why the very first attempt with devise_for nested under the namespace :api, :v1 didn't work because the final solution's logic is the same as the very first one's. The only difference is that in the second solution (the one without devise_scope), I was passing the /api/v1 to the path option instead of using nesting.

I will dig a little bit deeper into this and open the issue with the devise as I believe it might be a bug.

Nevertheless, those of you who were reading this article now know that there are two ways (at least that I know of :D) to configure custom routes for devise and make it work with JSON.

The key points to remember here are the following:

1. sign_up is handled by Devise::RegistrationController & sign_in/sign_out are handled by Devise::SessionsController
2. override registration and sessions controllers by adding respond_to :json to accept and return JSON
3. use devise_scope block to specify routes and controllers within a namespace block
4. alternatively use devise_for to specify controllers and use path: option to specify routes

Have fun coding 😉