Exchange Server 2013 CU1, two mailbox servers (DAG, only one mail database), two CAS servers (NLB cluster). Two Sendmail/CentOS-based SMTP relays route mail between Exchange servers and Internet (incoming mail is routed to NLB cluster name).

Usually mail flow is OK: messages are sent and received normally. However, I've got several complains from different users. They state that occasionally they don't receive messages that were sent to them from outside. Tomorrow morning I've decided to investigate the issue.

So, we have a message sent from Gmail account that was not delivered in user mailbox. It was received and routed to Exchange by Sendmail host:

Jun 27 22:06:37 MX01 sendmail[12169]: r5RG6HC8012166: to=<USER_ADDRESS>, delay=00:00:19, xdelay=00:00:19, mailer=smtp, pri=124702, relay=NLB-FQDN [NLB-IP], dsn=2.0.0, stat=Sent (<CAHgK_sjEkj41htuVsutMP3XhgR5RRDs8-EgSC-UfHoaiHWFj=A@mail.gmail.com> [InternalId=13069585481819] Queued mail for delivery) 

Running Get-MessageTrackingLog applet on the mailbox servers displayed the followed log entries for the message:

DB Server #1:

    22:11:18 HAREDIRECT

    22:11:18 RECEIVE

    22:11:18 AGENTINFO

DB Server #2:

    22:11:18 HARECEIVE

    22:12:44 HADISCARD

So, we can see that the message was placed into shadow queues, but was NOT delivered into mailbox (no DELIVER status entries). Also pay attention to the timing. Time on all the servers is in sync, but Sendmail timestamp is 22:06, and Exchange timestamps are 5 minutes behind it.

Today a message sent to the same address from the same Google mailbox was received by Exchange and placed into user mailbox with no glitches.

We use Exchange antispam system activated on Exchange mailbox servers. Spam is never dropped silently. It is either returned to sender (SCL 9) or placed into quarantine mailbox (SCL 6 and above). I'm pretty sure that the lost message was not in the quarantine mailbox this morning, and the sender didn't receive any NDRs.

Any ideas what to investigate next?

Hi,

Do you mean the emails are placed in the Shadow Redundancy queue?

Shadow Redundancy

http://technet.microsoft.com/en-us/library/dd351027(v=exchg.150).aspx

Thanks,

If you have feedback for TechNet Subscriber Support, contact tnsfl@microsoft.com

Simon,

I mean that the messages are DISAPPEARING. The only traces of them I can find are statuses related to shadow redundancy queues.

Hi,

We can enable the pipeline tracing log to find whats the problematic server that causes this issue.

Pipeline Tracing

http://technet.microsoft.com/en-us/library/bb125018(v=exchg.150).aspx

Thanks,

. If you have feedback for TechNet Subscriber Support, contact tnsfl@microsoft.com

Simon,

Pipeline tracing captures copies of email messages from a specific sender. I have messages reported as lost that had been sent by different random senders. Other messages sent by the same senders are delivered properly. So pipeline tracing is useless in my case.