Sequel::Model's quite optional - it doesn't force you to use the AR pattern. It's built on top of Sequel::Dataset, which is entirely usable all on its own.
It doesn't force you, but you lose all of it's ORM features. Sequel::Dataset is basically just a query builder so ROM would probably be a better choice then yeah.