Previously, email replies were handled inside workers. There was no
execution logs. This meant if emails silently failed (as reported by a
customer), we had no way to trace where the issue happened, the only
assumption was “no error = mail sent.”
By moving email handling into jobs, we now have proper execution logs
for each attempt. This makes it easier to debug delivery issues and
would have better visibility when investigating customer reports.
Fixes
https://linear.app/chatwoot/issue/CW-5538/emails-are-not-sentdelivered-to-the-contact
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
Changes searchkick callback behavior to check `should_index?` before
creating reindex jobs, preventing unnecessary job creation for messages
that don't need indexing (activity messages, unpaid accounts, etc.).
Previously, `callbacks: :async` created reindex jobs for all messages
(~5,100/min or 7.3M/day in production), which were then filtered by
`should_index?` inside the job worker - resulting in 98% wasted jobs,
Redis memory pressure, and avoidable p0 alerts.
Now, `should_index?` is checked before job creation via `after_commit`
callback, reducing job creation to actual incoming/outgoing messages
from paid accounts.
Changes:
- Disable automatic searchkick callbacks
- Add manual `after_commit` callback with `should_index?` condition
- Add specs to verify callback behavior
Expected impact:
- 98% reduction in sidekiq job creation (~7.3M → ~150K jobs/day)
- Reduced redis memory usage
- Same async indexing behavior for eligible messages
With this change, the indexing would be separate from the search, so you
need to enable indexing on the cloud and run it. It should start
indexing the messages to ElasticSearch/OpenSearch. Once indexing is
completed, we can turn on the feature for the customer.
Make sure that the following is done when you deploy.
Set POSTGRES_STATEMENT_TIMEOUT=600s before you run the indexing.
1. Make sure that the account with advanced_search has
advanced_search_indexing enabled
```rb
Account.feature_advanced_search.each do |account|
account.enable_features(:advanced_search_indexing)
account.save!
end
```
2. Enable indexing for all accounts with paid subscription.
```rb
Account.where("custom_attributes ->> 'plan_name' IN (?)", ['Enterprise', 'Startups', 'Business']).each do |account|
account.enable_features(:advanced_search_indexing)
account.save!
end
```
3. Run indexing for all the messages.
```rb
Message.reindex
```
Co-authored-by: Vishnu Narayanan <iamwishnu@gmail.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
We now support searching within the actual message content, email
subject lines, and audio transcriptions. This enables a faster, more
accurate search experience going forward. Unlike the standard message
search, which is limited to the last 3 months, this search has no time
restrictions.
The search engine also accounts for small variations in queries. Minor
spelling mistakes, such as searching for slck instead of Slack, will
still return the correct results. It also ignores differences in accents
and diacritics, so searching for Deja vu will match content containing
Déjà vu.
We can also refine searches in the future by criteria such as:
- Searching within a specific inbox
- Filtering by sender or recipient
- Limiting to messages sent by an agent
Fixes https://github.com/chatwoot/chatwoot/issues/11656
Fixes https://github.com/chatwoot/chatwoot/issues/10669
Fixes https://github.com/chatwoot/chatwoot/issues/5910
---
Rake tasks to reindex all the messages.
```sh
bundle exec rake search:all
```
Rake task to reindex messages from one account only
```sh
bundle exec rake search:account ACCOUNT_ID=1
```
The term "sorcerer’s apprentice mode" is defined as a bug in a protocol
where, under some circumstances, the receipt of a message causes
multiple messages to be sent, each of which, when received, triggers the
same bug. - RFC3834
Reference: https://github.com/chatwoot/chatwoot/pull/9606
This PR:
- Adds an auto_reply attribute to message.
- Adds an auto_reply attribute to conversation.
- Disable conversation_created / conversation_opened event if auto_reply
is set.
- Disable message_created event if auto_reply is set.
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
We made so many improvements for CSAT via https://github.com/chatwoot/chatwoot/pull/11485. However, we missed showing message content in the dashboard for CSAT URLs created in non-widget inboxes. This PR fixes the issue by ensuring that CSAT-configured messages are passed along with CSAT responses, otherwise defaulting to the translation.
The `before_type_cast` method sometimes returns a string for
`message_type`, creating inconsistencies in different payloads. This
pull request will remove all `before_type_cast` usage and replace it
with `to_i` methods.
The reload method in our callback was refreshing the object and hence the saved_change_to_assignee_id? Method wasn't working in the following callbacks.
This impacted the listeners subscribing to the event `ASSIGNEE_CHANGE`, `TEAM_CHANGE` etc
Updating the `unattended` tab to include conversations where the customer responded and is awaiting an agent's response.
Previously it showed only the conversations where the first response was pending.
Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
* logging the messages id for message validation exception
* Update the processed_message_content validation over length
* codeclimate
* specs failing for contacts
When a user mentions the connected Instagram page in a story, the story's content is downloaded in Chatwoot, then if the user deletes the story, the content persists in the platform.
fixes: #5258
Automate conversation display_id generation with db triggers
Co-authored-by: Saurabh Mehta <saurabh1.mehta@airtel.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
* [#139] Delayed emails for conversations
* Added the setex and get methods to Redis wrapper
* Set the priorities for the sidekiq queues
* Was not able to use mailhog for testing email in local, switched back to letter opener and added comments on using the SMTP settings
* Added after create hood in messages to queue the sending of mail after 2 minutes using sidekiq worker and also set the redis key for the conversation to avoid the email sending for every message
* Added the sidekiq worker to send the email and delete the conversation redis key
* Added the mailer and mail template
* mailer sends the last 10 messages along with the new messages from the time it was queued
* Send email only in development or if smtp config is set
* Send email only in development or if smtp config is set
* Set the SMTP_PORT in production variable
* Adding redis to circle CI
* Specs for the conversation email changes
* Added specs for conversation email sidekiq worker
* Added specs for conversation mailer
* Added specs in message model for the after create hook for notify email
* Send emails only when there is a reply from agent
* set development to use mailhog
* Adding comments for using letter opener
