## Context
Sidekiq logs only showed the Sidekiq wrapper class and JID, which wasn’t
helpful when debugging ActiveJobs.
## Changes
- Updated `ChatwootDequeuedLogger` to log the actual `ActiveJob class`
and `job_id` instead of the generic Sidekiq wrapper and JID.
> Example
> ```
> Dequeued ActionMailer::MailDeliveryJob
123e4567-e89b-12d3-a456-426614174000 from default
> ```
- Remove sidekiq worker and unify everything to `ActiveJob`
Previously, email replies were handled inside workers. There was no
execution logs. This meant if emails silently failed (as reported by a
customer), we had no way to trace where the issue happened, the only
assumption was “no error = mail sent.”
By moving email handling into jobs, we now have proper execution logs
for each attempt. This makes it easier to debug delivery issues and
would have better visibility when investigating customer reports.
Fixes
https://linear.app/chatwoot/issue/CW-5538/emails-are-not-sentdelivered-to-the-contact
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
## Linear:
- https://github.com/chatwoot/chatwoot/issues/486
## Description
This PR implements Multi-Factor Authentication (MFA) support for user
accounts, enhancing security by requiring a second form of verification
during login. The feature adds TOTP (Time-based One-Time Password)
authentication with QR code generation and backup codes for account
recovery.
## Type of change
- [ ] New feature (non-breaking change which adds functionality)
## How Has This Been Tested?
- Added comprehensive RSpec tests for MFA controller functionality
- Tested MFA setup flow with QR code generation
- Verified OTP validation and backup code generation
- Tested login flow with MFA enabled/disabled
## Checklist:
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
We now support searching within the actual message content, email
subject lines, and audio transcriptions. This enables a faster, more
accurate search experience going forward. Unlike the standard message
search, which is limited to the last 3 months, this search has no time
restrictions.
The search engine also accounts for small variations in queries. Minor
spelling mistakes, such as searching for slck instead of Slack, will
still return the correct results. It also ignores differences in accents
and diacritics, so searching for Deja vu will match content containing
Déjà vu.
We can also refine searches in the future by criteria such as:
- Searching within a specific inbox
- Filtering by sender or recipient
- Limiting to messages sent by an agent
Fixes https://github.com/chatwoot/chatwoot/issues/11656
Fixes https://github.com/chatwoot/chatwoot/issues/10669
Fixes https://github.com/chatwoot/chatwoot/issues/5910
---
Rake tasks to reindex all the messages.
```sh
bundle exec rake search:all
```
Rake task to reindex messages from one account only
```sh
bundle exec rake search:account ACCOUNT_ID=1
```
Rails 7.1 ships with connection-pooling enabled by default for
`RedisCacheStore` (see rails/rails#45235).
Because we already wrap our Redis clients in our own `ConnectionPool`
($alfred / $velma), the upgrade resulted in a double-wrapped object and
runtime errors such as:
NoMethodError: undefined method `get` for an instance of ConnectionPool
This patch:
* Passes `pool: false` when instantiating `RedisCacheStore` in
`config/initializers/rack_attack.rb`, telling Rails to use the pool we
supply instead of building its own.
* Adds an inline comment explaining the rationale.
* Adds a TODO in `config/initializers/01_redis.rb` suggesting a future
simplification: switch to plain Redis clients and let Rails manage the
pool.
Reference docs:
* rails/rails#45235 – “Enable connection pooling by default for
MemCacheStore and RedisCacheStore” -
https://github.com/rails/rails/pull/45235
* Rails 7.1 Caching Guide – 2.1.1 “Connection Pool Options” (use `pool:
false`) [Ruby on Rails
Guides](https://guides.rubyonrails.org/v7.1/caching_with_rails.html)
Migration Guide: https://chwt.app/v4/migration
This PR imports all the work related to Captain into the EE codebase. Captain represents the AI-based features in Chatwoot and includes the following key components:
- Assistant: An assistant has a persona, the product it would be trained on. At the moment, the data at which it is trained is from websites. Future integrations on Notion documents, PDF etc. This PR enables connecting an assistant to an inbox. The assistant would run the conversation every time before transferring it to an agent.
- Copilot for Agents: When an agent is supporting a customer, we will be able to offer additional help to lookup some data or fetch information from integrations etc via copilot.
- Conversation FAQ generator: When a conversation is resolved, the Captain integration would identify questions which were not in the knowledge base.
- CRM memory: Learns from the conversations and identifies important information about the contact.
---------
Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
- Throttle reports API requests at the account level
- Throttle reports API requests at the user level for dashboard users as
well as API users
Co-authored-by: Sojan Jose <sojan@pepalo.com>
40 % of Chatwoot's current log volume is from state transition logs generated by `rack-timeout`, which are logged at the `INFO` level. This PR reduce the noise in logs and set RACK::TIMEOUT to log at `error` level
The current task for loading `GeoLite2-City.mmdb` doesn't work for all build types. This PR addresses this and move the task to initializer to ensure consistency across environments.
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Sojan Jose <sojan.official@gmail.com>
This PR addresses several issues related to logging:
- Enabling Lograge broke certain non-API URLs, such as password reset.
This occurred due to the user ID tagging we had in Lograge, which has
now been limited to API pages only.
- Disabled the start and done logs in Sidekiq.
- Investigated why Sidekiq logs weren’t being output as JSON. This is
due to the use of ActiveJob instead of Sidekiq for the job base classes.
**Potential Options for Converting ActiveJob Logs to JSON:**
- https://glozer.rocks/ojb
- https://learnedreverie.medium.com/activejob-logs-as-json-6912403d8c81
- https://github.com/roidrage/lograge/pull/226
- Fix git-hash in super admin instance health page for Heroku installations
- Fix git boundary not set error for docker/heroku deployments which remove .git directory
This PR refactors the schema we introduced in #7518 based on the feedback from production tests. Here is the change log
- Decouple Inbox association to a new table inbox_response_sources -> this lets us share the same response source between multiple inboxes
- Add a status field to responses. This ensures that, by default, responses are created in pending status. You can do quality assurance before making them active. In future, this status can be leveraged by the bot to auto-generate response questions from conversations which require a handoff
- Add response_source association to responses and remove hard dependency from response_documents. This lets users write free-form question answers based on conversations, which doesn't necessarily need a response source.
- The message sender was incorrect. Incoming messages were created under the contact's name, and outgoing messages were created under the user's name instead of the reverse.
- The seed user's email address was incorrect in the message data.
- The Sendmail configuration overrode the Letter Opener config which made it difficult to test the email.
- This PR also fixes an ESLint lint issue on develop.
This commit introduces the ability to associate response sources to an inbox, allowing external webpages to be parsed by Chatwoot. The parsed data is converted into embeddings for use with GPT models when managing customer queries.
The implementation relies on the `pgvector` extension for PostgreSQL. Database migrations related to this feature are handled separately by `Features::ResponseBotService`. A future update will integrate these migrations into the default rails migrations, once compatibility with Postgres extensions across all self-hosted installation options is confirmed.
Additionally, a new GitHub action has been added to the CI pipeline to ensure the execution of specs related to this feature.
- Add lograge gem to improve rails logging using `LOGRAGE_ENABLED` env variable
- When enabled Single line log for requests in JSON formatting
- Switch sidekiq also to use JSON formatting
Fixes: chatwoot/product#437
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
