While investigating a customer-reported issue, I found that some emails
were appearing late in Chatwoot. The root cause was query timeouts.
It only happened for emails with an in_reply_to header. In these cases,
Chatwoot first checks if a message exists with message_id = in_reply_to.
If not, it falls back to checking conversations where
additional_attributes->>'in_reply_to' = ?.
We were using:
```rb
@inbox.conversations.where("additional_attributes->>'in_reply_to' = ?", in_reply_to).first
```
This looked harmless, but .first caused timeouts. Without .first, the
query ran fine. The issue was the generated SQL:
```sql
SELECT *
FROM conversations
WHERE inbox_id = $1
AND (additional_attributes->>'in_reply_to' = '<in-reply-to-id>')
ORDER BY id ASC
LIMIT $2;
```
The ORDER BY id forced a full scan, even with <10k records.
The fix was to replace .first with .find_by:
```rb
@inbox.conversations.find_by("additional_attributes->>'in_reply_to' = ?", in_reply_to)
```
This generates:
```sql
SELECT *
FROM conversations
WHERE inbox_id = $1
AND (additional_attributes->>'in_reply_to' = '<in-reply-to-id>')
LIMIT $2;
```
This avoids the scan and runs quickly without needing an index.
By the way, Cursor and Claude failed
[here](https://github.com/chatwoot/chatwoot/pull/12401), it just kept on
adding the index without figuring out the root cause.
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
The term "sorcerer’s apprentice mode" is defined as a bug in a protocol
where, under some circumstances, the receipt of a message causes
multiple messages to be sent, each of which, when received, triggers the
same bug. - RFC3834
Reference: https://github.com/chatwoot/chatwoot/pull/9606
This PR:
- Adds an auto_reply attribute to message.
- Adds an auto_reply attribute to conversation.
- Disable conversation_created / conversation_opened event if auto_reply
is set.
- Disable message_created event if auto_reply is set.
---------
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Add bounced emails to the conversation thread.
Fix Gmail bounce detection by checking the X-Failed-Recipients header.
Currently, bounced emails are rejected as auto-replies, which causes
support agents to miss important delivery failure context. This PR
ensures bounced messages are correctly added to the thread, preserving
visibility for the support team.
In Chatwoot, we rely on the Content-ID for inline attachments to replace
the link with the uploaded attachment URL. Our expectation was that only
images would be inline, while other attachments would not. However,
email clients like Apple Mail (sigh) allow users to send inline
attachments that are not images, and these attachments often lack a
Content-ID. This creates significant issues in rendering.
I investigated how other email clients handle this scenario. When
viewing the same email (sent from Apple Mail) in Gmail, only one image
appears—and it’s treated as an attachment, not inline. This happens
because both attachments are the same image, and Apple Mail only sends
one copy. See the screenshot below.
| Apple Mail | Gmail |
| -- | -- |
| <img width="646" alt="Screenshot 2025-02-27 at 8 20 17 PM"
src="https://github.com/user-attachments/assets/e0d1cd2d-e47c-4081-a53b-7a67106341b3"
/> | <img width="360" alt="Screenshot 2025-02-27 at 8 20 51 PM"
src="https://github.com/user-attachments/assets/b206e56e-8f86-43e9-867b-d895c36aff78"
/> |
A good fix for this would be to check if the Content-ID is missing and
then upload the file as a regular attachment. However, the Mail gem (for
some reason) automatically adds a default Content-ID to inline parts. I
need to dig into the source code to understand why this happens.
For now, I’ve implemented a check to treat non-image attachments as
regular attachments. Inline image attachments are already handled by
appending an image tag at the end if the content-id is not found in the
body. A sample conversation to test this behavior is
[here](https://app.chatwoot.com/app/accounts/1/conversations/46732).
Some email clients automatically set Content-Disposition to inline for
specific content types, such as images. In cases where the email body is
empty, inline attachments may not display correctly due to our previous
implementation. Our assumption was that these attachments are referenced
within text/plain or text/html parts.
Customer-reported issues, especially with Apple Mail, show emails with
attachments marked as inline but without any corresponding text parts.
This leads to missing attachments even though would have processed the
attachment.
This update introduces a check for the presence of a text part. If none
exists, inline attachments are treated as regular attachments and added
to the external attachments array, ensuring that all attachments display
properly.
<details>
<summary><b>Script to update the existing emails that are already
available in the system</b></summary>
```rb
def update_content id
message = Message.find id
conversation = message.conversation
message_id = message.source_id
channel = message.inbox.channel
authentication_type = 'XOAUTH2'
imap_password = Google::RefreshOauthTokenService.new(channel: channel).access_token
imap = Net::IMAP.new(channel.imap_address, port: channel.imap_port, ssl: true)
imap.authenticate(authentication_type, channel.imap_login, imap_password)
imap.select('INBOX')
results = imap.search(['HEADER', 'MESSAGE-ID', message_id])
message_content = imap.fetch(results.first, 'RFC822').first.attr['RFC822']
mail = MailPresenter.new(Mail.read_from_string(message_content))
mail_content = if mail.text_content.present?
mail.text_content[:reply]
elsif mail.html_content.present?
mail.html_content[:reply]
end
attachments = mail.attachments.last(Message::NUMBER_OF_PERMITTED_ATTACHMENTS)
inline_attachments = attachments.select { |attachment| attachment[:original].inline? && mail_content.present? }
regular_attachments = attachments - inline_attachments
regular_attachments.each do |mail_attachment|
attachment = message.attachments.new(
account_id: conversation.account_id,
file_type: 'file'
)
attachment.file.attach(mail_attachment[:blob])
end
message.save!
end
```
</details>
* fix: downcase email when finding
* feat: add `from_email` class
* refactor: use `from_email`
* feat: add rule to disallow find_by email directly
* chore: remove redundant test
Since the previous imlpmentation didn't do a case-insentive search, a new user would be created, and the error would be raised at the DB layer. With the new changes, this test case is redundant
* refactor: use from_email
* fix: change email conversation not found exception to log
* chore: refactor reply_mailbox methods
---------
Co-authored-by: Sojan Jose <sojan@pepalo.com>
Refactor code in ApplicationMailbox
* short-circuiting as soon as we get a "true" value in some cases
* using ".exists?" instead of instantiating an ActiveRecord object
* using "match?" instead of "match"
This change allows the user to configure both IMAP and SMTP for an email inbox. IMAP enables the user to see emails in Chatwoot. And user can use SMTP to reply to an email conversation.
Users can use the default settings to send and receive emails for email inboxes if both IMAP and SMTP are disabled.
Fixes#2520
Configuring an agent email also as a support email inbox leads to conversations getting created in a loop if notifications were also configured to the same email.
For emails forwarded from google groups, Google rewrites the FROM address to the group email and the original email will be available under X-Original-Sender. This PR enables chatwoot to handle this case.
Fixes: #2715
* feat: added support mailbox to handle email channel (#140)
Added a new mailbox called 'SupportMailbox' to handle all the
incoming emails other than reply emails.
An email channel will have a support email and forward email
associated with it. So we filter for the right email inbox based on
the support email of that inbox and route this to this mailbox.
This mailbox finds the account, inbox, contact (create a new one
if it does not exist) and creates a conversation and adds the
email content as the first message in the conversation.
Other minor things handled in this commit:
* renamed the procs for routing emails in application mailbox
* renamed ConversationMailbox to ReplyMailbox
* Added a fallback content in MailPresenter
* Added a record saving (bang) versions of enabling and disabling
features in Featurable module
* added new factory for the email channel
refs: #140
