mirror of
https://github.com/lingble/chatwoot.git
synced 2025-11-03 20:48:07 +00:00
fdbb3bf4b1
The Inboxes::FetchImapEmailsJob is designed to fetch the entire email object and check if its message id is already in the database. However, this process is resource-intensive and time-consuming, as fetching the full email object takes a significant amount of time. On average, fetching 100 emails can take approximately 3-4 minutes to complete depending on the IMAP server. Since we are not using server flags to identify which emails have already been fetched (to avoid compatibility issues with flags in different email services), we have to fetch all previously available emails. Currently we fetch all the messages that were created from yesterday. This becomes problematic with services like Gmail, which throttle requests based on bandwidth usage. To address this issue, I have updated the logic as follows: Fetch the sequence IDs of all the mails in the "Inbox" that were created from yesterday. Use the FETCH command to fetch only the message-ids using BODY.PEEK[HEADER.FIELDS (MESSAGE-ID)] with the sequence IDs we got in the previous step. This is a faster operation with lower bandwidth usage, as it only returns the sequence ID and message ID. Check if the message IDs are already present in the database for the selected inbox. If not present, fetch the entire email object using the FETCH command and create the message. If the message ID is already present, ignore it and move on to the next message-id. I have also addressed the issue of duplicate emails appearing in conversations when two fetch email jobs occur simultaneously. I have added a lock for the channel to ensure that the job gracefully exits without waiting for the current job to complete. Fixes #7247 Fixes #6082 Fixes #8314 Co-authored-by: Sojan <sojan@pepalo.com>
43 lines
2.1 KiB
Ruby
43 lines
2.1 KiB
Ruby
module Redis::RedisKeys
|
|
## Inbox Keys
|
|
# Array storing the ordered ids for agent round robin assignment
|
|
ROUND_ROBIN_AGENTS = 'ROUND_ROBIN_AGENTS:%<inbox_id>d'.freeze
|
|
|
|
## Conversation keys
|
|
# Detect whether to send an email reply to the conversation
|
|
CONVERSATION_MAILER_KEY = 'CONVERSATION::%<conversation_id>d'.freeze
|
|
# Whether a conversation is muted ?
|
|
CONVERSATION_MUTE_KEY = 'CONVERSATION::%<id>d::MUTED'.freeze
|
|
|
|
## User Keys
|
|
# SSO Auth Tokens
|
|
USER_SSO_AUTH_TOKEN = 'USER_SSO_AUTH_TOKEN::%<user_id>d::%<token>s'.freeze
|
|
|
|
## Online Status Keys
|
|
# hash containing user_id key and status as value
|
|
ONLINE_STATUS = 'ONLINE_STATUS::%<account_id>d'.freeze
|
|
# sorted set storing online presense of account contacts
|
|
ONLINE_PRESENCE_CONTACTS = 'ONLINE_PRESENCE::%<account_id>d::CONTACTS'.freeze
|
|
# sorted set storing online presense of account users
|
|
ONLINE_PRESENCE_USERS = 'ONLINE_PRESENCE::%<account_id>d::USERS'.freeze
|
|
|
|
## Authorization Status Keys
|
|
# Used to track token expiry and such issues for facebook slack integrations etc
|
|
AUTHORIZATION_ERROR_COUNT = 'AUTHORIZATION_ERROR_COUNT:%<obj_type>s:%<obj_id>d'.freeze
|
|
REAUTHORIZATION_REQUIRED = 'REAUTHORIZATION_REQUIRED:%<obj_type>s:%<obj_id>d'.freeze
|
|
|
|
## Internal Installation related keys
|
|
CHATWOOT_INSTALLATION_ONBOARDING = 'CHATWOOT_INSTALLATION_ONBOARDING'.freeze
|
|
LATEST_CHATWOOT_VERSION = 'LATEST_CHATWOOT_VERSION'.freeze
|
|
# Check if a message create with same source-id is in progress?
|
|
MESSAGE_SOURCE_KEY = 'MESSAGE_SOURCE_KEY::%<id>s'.freeze
|
|
OPENAI_CONVERSATION_KEY = 'OPEN_AI_CONVERSATION_KEY::%<event_name>s::%<conversation_id>d::%<updated_at>d'.freeze
|
|
|
|
## Sempahores / Locks
|
|
# We don't want to process messages from the same sender concurrently to prevent creating double conversations
|
|
FACEBOOK_MESSAGE_MUTEX = 'FB_MESSAGE_CREATE_LOCK::%<sender_id>s::%<recipient_id>s'.freeze
|
|
IG_MESSAGE_MUTEX = 'IG_MESSAGE_CREATE_LOCK::%<sender_id>s::%<ig_account_id>s'.freeze
|
|
SLACK_MESSAGE_MUTEX = 'SLACK_MESSAGE_LOCK::%<conversation_id>s::%<reference_id>s'.freeze
|
|
EMAIL_MESSAGE_MUTEX = 'EMAIL_CHANNEL_LOCK::%<inbox_id>s'.freeze
|
|
end
|