Ruby on Rails modularization with Packwerk Episode II

Application as a package

Create the structure

First, we need to create app/packages folder where we will place all our packages. In order to isolate our packages we have to separate every MVC concept in one folder. Taking the CodeTriage project as an example we will have something like the following picture.

If we try to run the server, it will fail to find the constants. That's why we need to add a line of configuration to our application.rb

config.paths.add 'app/packages', glob: '*/{*,*/concerns}', eager_load:true

Now the application works but it cannot find the views, so we need to add another line of configuration to our application_controller.rb

append_view_path(Dir.glob(Rails.root.join('app/packages/*/views')))

Create the packages

Our structure is ready, so now we can start creating the packages. In order to do that, we only need to add apackage.yml to every folder with the following configuration:

enforce_privacy: false 
enforce_dependencies: true

enforce_privacygives us the possibility to isolate all the constants of the package and work with a public API. In order to expose the public constants, we need to add the constants in, for example packages/users/app/public.For now we are going to set this configuration to false.

enforce_dependencies will enforce the dependency of a package and check for all the constant references. If a dependency is not explicitly defined it will be a violation of the boundary.

Validating the package system

Packwerk established a criterion we need to follow in order to have a valid package system. We can start running packwerk validate in our console.

 This will check our folder structure, package configuration, and autoload path cache.

Right now, our application is not valid and we have to fix the load paths in**packwerk.yml.** In order to do this, we only have to add the missing paths.

# packwerk.yml 
 
load_paths: 
. 
. 
. 
 
# Users 
- app/packages/users/controllers 
- app/packages/users/models 
-  app/packages/users/package.yml
- app/packages/users/views

At this point, we are ready to check boundary violations in our application. To check violations we can runpackwerk update-deprecations , this command will generate deprecated_references.yml file for every package. In every file, we will find package name, type of violation, and file path. With all this information we know where the violation is happening and we can make a decision to resolve it.

# deprecated_references.yml 
 
. 
. 
. 
 
app/packages/repos: 
  "::Repo": 
    violations: 
    - dependency 
    files: 
    - app/packages/users/models/user.rb

Taking the example we are going to describe every part of the information generated
by Packwerk.

- app/packages/repos  - package where the constant violation is found.\

- ::Repo  - path to the file containing the violated constant.

- dependency  - a type of violation, either dependency or privacy.

- app/packages/users/models/user.rb  - path to the file containing the violated constant.

As a final step in this section, don't forget to add the new generated file paths to packwerk.yml and run validations again.

Dependency visualization

With all the information in package.yml and deprecated_references.ymlwe can then
visualize a graph of dependencies. In order to do that we need to add another gem, in this case we will use Pocky.

Running rake pocky:generate we will generate a file called packwerk.png where we can visualize our first graph of dependencies.

With all the packages defined our graph will look like this.

dependencies already exist but it doesn't mean they are accepted by Packwerk. To
accept a dependency we need to add dependencies configuration to the package.yml
in every package. We will focus on mail_builders since it's a package without circular dependency. It's worth mentioning that Packwerk won't let us accept circular dependencies.

# app/packages/mail_builders/package.yml 

```ruby
enforce_privacy: false 
enforce_dependencies: true 
dependencies: 
- app/packages/docs 
- app/packages/issues 
- app/packages/repos

After adding this configuration, Pocky will color the accepted dependencies using green.

We can delete deprecated_references.yml from app/packages/mail_builders and run
packwerk update-deprecations again. The file won't be generated again since all the
violations were fixed for this package. It's important to mention that even if we don't Graph with accepted dependencies

Ruby on Rails modularization with Packwerk accept dependencies our application will still work as before, but now we have more
information to take decisions and refactor.

Remove circular dependencies

In our previous graph, we had a lot of circular dependencies that needed to be resolved somehow. We have different strategies to do that:

- Do nothing,

- Accept dependencies, Merge packages,

- Move code between packages,

- Duplicate a functionality, 

- Perform dependency injection or Dependency injection with typing.

One issue here is that in order to do a proper refactor, we need to know the codebase. I'm not so familiar with the codebase of this project since I took it as an example, so for practical reasons we will go with the first strategy, do nothing. Even if we will avoid most of the refactoring, we want to work on the dependencies in the root package.

The root package contains all the glue from the Rails framework, all the classes we inherit from and make all work together. So, in order to solve the circular dependencies, we are going to create a new package called rails within the following steps:

1. Move all the application_ files and folders from the app to app/packages/rails.
2. Create apackage.yml for the package with the same configuration as the previous packages.
3. Add all the new file paths to packwerk.yml.
4. Add app/packages/rails as a dependency from the rest of the packages.

Once we create the package we will start to notice a lot of files that can be re-structured. After moving everything to the corresponding package and accepting
dependencies we will have a new structure and a cleaner graph.

Remove dependencies from the root package

Now our graph looks much better would be great if we can remove all the dependencies from root package. If we check deprecated_references.yml in the root package, we will notice that most of them are from test , lib/tasks , db and config
folder. In order to resolve these dependencies, we are going to create a test folder within every package. Having something like app/packages/users/test. Next, we are going to exclude lib/tasks , db and configamong other folders from Packwerk analysis since those dependencies aren't really important in our analysis and we don't have an easy way to resolve them. We will add the following to our packwerk.yml.

exclude: 
- "{bin,node_modules,script,tmp,vendor,lib,db,config,perf_scripts}/**/*" 
- "lib/tasks/**/*.rake"

After moving all the tests from root package and excluding the folders from the analysis we will have a new graph without root dependencies.

As we can see, we still have circular dependencies inusers , repos , and docs . Although we didn't resolve them, we have important information to pass now. We know that every team that performs changes in one of those packages will probably have to perform changes in the packages with the circular dependency. On the other hand, we know that a team can work on github_fetchers solely, knowing what packages are
getting affected with the changes in every moment.

You can find the final result of the project here.

Next step

As a next step, you could enforce constant privacy in every package and expose only the public API that will be accessible from other packages. You can easily configure where your API will be placed in package.yml.

enforce_privacy: true 
enforce_dependencies: true 
public_path: my/custom/path/

Conclusions

Packwerk gives us a lot of information about our application and with that information we can take decisions to improve the workflow of our teams. Although the process seemed to be long and with a lot of configurations, it doesn't have to be like that always. We can start creating packages only for the new code added to our application and then modularize gradually. So now we can start to talk about Gradual Modularization this is the concept introduced by Stephan Hagemann "We can, for the first time, decide to start modularizing a portion of the code in an aspirational way... This allows us to create a gradually expanding support system towards a better application structure".

Sources

1. Gradual Modularization for Ruby on Rails - Stephan Hagemann
2. Enforcing Modularity in Rails Apps with Packwerk
3. Packwerk Github
4. Source Code of the Article

Read More

GraphQL Ruby. What about performance?

Rails and Other Means of Transport

Rails Development with TMUX, Vim, Fzf + Ripgrep