Generating definitions for custom ruby app configuration

[johnernaut]

Hello,

I have a custom Ruby app that I'd like to generate RBI files for. I do use active record and an app/models directory structure, but I'm handling loading myself in what's most likely a non-standard way, so I don't think that tapioca is able to find the files within my structure to create RBI definitions for them. My folder structure looks roughly like the following:

├── Gemfile
├── Gemfile.lock
├── app
│   ├── models
│   ├── services
│   └── workers
├── bin
│   ├── server
│   └── tapioca
├── config
│   ├── environment.rb
│   ├── initializers
├── db
│   ├── migrate
│   ├── seeds.rb
├── sorbet
│   ├── config
│   ├── rbi
│   └── tapioca

I updated my sorbet/tapioca/require.rb file to attempt to require each file in my app/models folder individually so tapioca could find them, but that doesn't generate any definitions for them (there are no errors or warnings either):

app_root = File.expand_path("../..", File.dirname(__FILE__))

require "active_record"
# require ...other things ...

Dir[File.expand_path "#{app_root}/app/models/**/*.rb"].sort.each do |file|
  require file
end

Dir[File.expand_path "#{app_root}/app/services/**/*.rb"].sort.each do |file|
  require file
end

Dir[File.expand_path "#{app_root}/app/workers/**/*.rb"].sort.each do |file|
  require file
end

I was wondering if you all had any ideas as to how I might generate those definitions based on the above. Thanks!

[Morriar]

👋 Hey @johnernaut,

We're working on a custom loader solution that, I think, would solve this problem.

[johnernaut]

@Morriar oh wow, awesome timing! I'll close this and keep an eye on that PR. Thanks!

[johnernaut]

@Morriar Just as a heads up, when I test out the new functionality described in the PR (and point my gemfile to that branch) using the same require code above, I get the following output from Tapioca:

No classes/modules can be matched for RBI generation.
Please check that the requested classes/modules include processable DSL methods.

I am still using ActiveRecord, it's just being done outside of a Rails app. Here's the full output if you're curious (I added logging to show the full file paths for what I'm requiring as well):

Loading my custom application application...
/Users/user/code/my-app/app/models/application_record.rb
/Users/user/code/my-app/app/models/team.rb
/Users/user/code/my-app/app/models/user.rb
 Done
Compiling DSL RBI files...

No classes/modules can be matched for RBI generation.
Please check that the requested classes/modules include processable DSL methods.

[Morriar]

While we're working on this, one hacky way to create a custom loader would be to define a custom DSL compiler under sorbet/tapioca/compilers so it get found by Tapioca when starting DSL generation and teach it how to load your app.

You even be able to trigger autoloads manually:

# sorbet/tapioca/compilers/load_my_custom_rails_app.rb

require "../../../app/config/environment"
Rails.autoloaders.each(&:eager_load)

[johnernaut]

@Morriar ah, interesting - thanks for looking into this! Really appreciate it. We don't actually use Rails at all we just have a similar structure and manually include our files in config/environment using Zeitwork. I may be able to do something similar to what you mentioned though: I'll give it a shot. Thanks again!

[johnernaut]

Wanted to share how I got this mostly working in our app with the structure defined in the original post and a config/enviornment.rb file that does some custom loading via Zeitwork. Here's what our config/environment.rb file looks like:

# many other things redacted
app_root = File.expand_path("..", File.dirname(__FILE__))

ENV["APP_ENV"] ||= "development"

require "rubygems"
require "bundler/setup"

Bundler.require(:default, ENV["APP_ENV"])

require "zeitwerk"

loader = Zeitwerk::Loader.new
loader.push_dir("#{app_root}/app/models")
loader.push_dir("#{app_root}/app/services")
loader.push_dir("#{app_root}/app/rpc/controllers")
loader.push_dir("#{app_root}/app/workers")

loader.setup

After creating a file in sorbet/tapioca/compilers/custom_app_loader.rb, I was able to get most of the app loaded and recognized by Tapioca. Unfortunately our app breaks with eager loading because we have some non-standard file / class namespaces due to auto-generated files that come from protobuf, but here's what the contents of that file look like:

require_relative "../../../config/environment"

Zeitwerk::Loader.eager_load_all

[AlexanderPavlenko]

@johnernaut thanks for the compilers/custom_app_loader.rb tip! works even without modifying config/environment.rb.

auto-generated files that come from protobuf

Managed to integrate them into Rails 7.1.1:

config/application.rb

  class Application < Rails::Application
    ...
    autoloaders.each do |autoloader|
      autoloader.inflector.inflect(
        'whatever_pb'          => 'Whatever',
        'whatever_services_pb' => 'Whatever')
    end
    $LOAD_PATH << "#{root}/grpc"
    config.autoload_once_paths << "#{root}/grpc"
    config.eager_load_paths << "#{root}/grpc"
    # Google::Protobuf::DescriptorPool fails on reload!

sorbet/tapioca/config.yml

...
dsl:
   exclude: # https://github.com/Shopify/tapioca/blob/main/manual/compilers.md
   - Protobuf

https://github.com/coinbase/protoc-gen-rbi#readme

go install github.com/coinbase/protoc-gen-rbi@latest
p=/rails/grpc
grpc_tools_ruby_protoc \
  --proto_path=$p --ruby_out=$p --grpc_out=$p \
  --rbi_out=$p/../sorbet/rbi/grpc $p/whatever.proto