Repository navigation
Allow admin to configure to be able to ignore "Connection timed out" errors shown in console #495
Description
Summary
When a sender's TCP connection drops at the network level, the TCP Listener logs the read failure at ERROR, with a full stack trace. This shows up in mirth.log, the console and the Administrator's Server Log panel. The disconnects heal themselves: the sender reconnects and resends, and nothing is lost. Even so, every drop looks like a real failure. I'd like a supported way to quiet these messages without also hiding the TCP errors that do matter.
Environment
- OIE 4.6.0, running on JDK 17
- About 40 inbound TCP Listener (MLLP) channels, each fed by a different external site
What we see
[2026-10-15 04:48:12.640] ERROR (com.mirth.connect.connectors.tcp.TcpReceiver:746): Error receiving message (TCP Listener "Source" on channel 72d28ead-8d2d-41af-a993-e83f178cdcf0).
java.net.SocketException: Connection timed out
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:330)
...
at com.mirth.connect.model.transmission.framemode.FrameStreamHandler.read(FrameStreamHandler.java:121)
at com.mirth.connect.connectors.tcp.TcpReceiver$TcpReader.readBytes(TcpReceiver.java:813)
at com.mirth.connect.connectors.tcp.TcpReceiver$TcpReader.call(TcpReceiver.java:620)
...
In one log window (20 rotated files of 500 KB each) there were 79 of these errors across about 40 channels:
| Exception | Count |
|---|---|
java.net.SocketException: Connection timed out |
42 |
java.net.SocketException: Socket closed |
37 |
The community has told me this is a network configuration issue somewhere between the sending sites and our server. It doesn't affect patient care and no data is lost, so I can't justify asking our network team, or the IT teams at every sending site, to chase it. The errors aren't going away, so we need a way to live with them without them cluttering the logs.
Why log4j alone isn't a clean fix
A RegexFilter on the TcpReceiver logger can suppress these errors (workaround below), but it's a blunt tool:
- The cause isn't in the log message. The message is always
Error receiving message (TCP Listener "…" on channel …)., and the actual cause (Connection timed out,Socket closed,Connection reset) appears only in the attached exception. log4j's built-in filters only look at the message text, so you can't suppress timeouts while keeping other read errors. - It applies to the whole server. You can't silence a channel whose sender is known to be flaky and keep full logging everywhere else.
- It's easy to lose. The file is per install, so every OIE upgrade means applying it again.
- Turning off the logger instead is worse. Setting the
TcpReceiverlogger toOFFalso hides real problems, such asBindException: Address already in usewhen a listener can't open its port.
Proposed solution
Any one of these would solve it. They're listed in order of preference:
- A TCP Listener setting, for example "Log connection-level read errors (timeout / reset / closed)", defaulting to on so current behaviour doesn't change. With it off, those errors would be logged at DEBUG or not at all. Errors that happen while a message is being parsed or processed would still log at ERROR.
- Log expected network drops at WARN (or INFO) instead of ERROR, without the stack trace. A peer dropping an idle connection is a normal event for a listener, not a server error.
- At minimum, put the cause in the message text, for example
Error receiving message (TCP Listener "Source" on channel …): Connection timed out. Admins could then filter on the specific cause with a standard log4jRegexFilter, with no code changes on their side.
Workaround (for anyone else hitting this)
Add the lines below to <OIE install>/conf/log4j2.properties and restart OIE. This suppresses only the Error receiving message (TCP Listener …) line. Other TcpReceiver errors, including bind failures, still log.
# Suppress self-healing inbound TCP read drops (peer/network idle timeouts).
# Matches only the "Error receiving message" line; other TcpReceiver errors still log.
logger.tcpReceiver.name = com.mirth.connect.connectors.tcp.TcpReceiver
logger.tcpReceiver.level = ERROR
logger.tcpReceiver.filter.readDrop.type = RegexFilter
logger.tcpReceiver.filter.readDrop.regex = .*Error receiving message \\(TCP Listener.*
logger.tcpReceiver.filter.readDrop.onMatch = DENY
logger.tcpReceiver.filter.readDrop.onMismatch = NEUTRALNotes:
RegexFilterhas to match the whole message, so the pattern needs.*at both ends. The\\(is the.propertiesescape for a literal(.- The filter is attached to the logger, so the event is dropped before it reaches any output:
mirth.log, the console and the Administrator's Server Log panel. - Limitation: this also hides
Socket closed,Connection resetand any other read error on every TCP Listener, because they all produce the same message text. That limitation is the reason for this feature request. - log4j only reads this file at startup, so it takes effect after a restart. To have later log4j changes applied without a restart, add
monitorInterval = 60at the top of the file. That setting itself needs one restart to take effect. - Add the lines again after every OIE upgrade, because a new version installs a fresh
conf/. - To check it's working: after the restart,
grep -c "Error receiving message" logs/mirth.logshould stay at 0 while other ERROR lines still appear.
Activity
pacmano1 commentedon Oct 17, 2026
A few things in here are off.
You can scope it to a channel. The channel ID is in the message, it's right there in the line you pasted. Put the flaky senders' channel IDs in the regex and every other channel keeps logging:
An upgrade doesn't replace
conf/. The installer keeps it, and the only files it removes are the ones the previous version installed. Anything you added yourself is left alone, whether it's incustom-lib, a resources folder or any other directory. I ran 4.5.2 to 4.6.0 over an existing install andlog4j2.propertieswas byte-for-byte the same afterwards. Nothing to re-add.The filter isn't what hides "Connection reset". That one is logged at WARN (
TcpReceiver.java:744in 4.6.0). The defaultlog4j2.propertiesonly shows ERROR, and so does your workaround withlevel = ERROR. It was never in your log to begin with.Turning the
TcpReceiverlogger off doesn't hide a bind failure.TcpReceivernever logs that one. The channel fails to start, andcom.mirth.connect.server.channel.ErrorTaskHandlerlogs it at ERROR asFailed to start channel ..., caused byBindException: Address already in use. I setlogger.tcpReceiver.level = OFFon 4.6.0, put a second listener on a port that was already taken, and got the same ERROR as with the default config. Turning the logger off is still a blunt tool, since it drops everything elseTcpReceiverlogs, just not for the reason you gave.It's not always harmless. A sender closing a connection normally logs nothing. "Connection timed out" means the OS gave up on a sender that went silent. One way to get there is the engine's ACK never reaching the sender: the engine records the message as received and sent with no error, but the sender never got its ACK, so if it resends, that's a duplicate. With Linux's default TCP settings that error shows up about 15 minutes after the message. If it shows up about 2h11m after the last message instead, the connection was idle and keepalive gave up on it, with no message in flight. Check which one you have before turning the logging off.
On the drafting side: we use AI on this project too, and most of the above could have been caught before filing. A few habits that help:
conf/" and "turning the logger off hides bind errors" were both assumptions. Anything marked assumed either gets tested or comes out.Set TCP keepalive idle time so idle connections survive NAT
tonygermano commentedon Oct 19, 2026
#502 should make many of these
Connection timed outerrors stop happening. It doesn't change how they're logged, so this issue stays open for that.What #502 changes. TCP Listener and TCP Sender connections that use keepalive now send the first probe after 120 seconds of idle time instead of the OS default, which is usually 2 hours. A probe every 2 minutes keeps the connection's entry alive in NAT devices and load balancers, which otherwise drop idle connections silently after a few minutes. The idle time is set with the
oie.tcp.keepalive.idlesystem property, in seconds, and 0 turns it off.The TCP Listener's "Keep Connection Open" setting is on by default, and that's what turns keepalive on, so MLLP listeners like yours are covered unless that setting was changed.
This changes the timing diagnostic above. With Linux's default probe settings (75 seconds apart, 9 probes), an idle connection whose peer really disappears now fails after about 13 minutes instead of about 2h11m. On a version with #502, an error about 13 minutes after the last message is the idle case, and that's close to the roughly 15 minutes of an undelivered ACK. Timing alone won't reliably tell them apart anymore. Instead, check whether the sender resent its last message after reconnecting: look for a second message with the same message control ID (MSH-10) shortly after the error. A duplicate means the sender never got its ACK.
Before upgrading, on Linux you can get the same effect for every socket on the host that uses keepalive with:
Add it to
/etc/sysctl.d/to make it survive a reboot. This affects every process on the host, not just the engine. In Docker the setting is per container, so pass--sysctl net.ipv4.tcp_keepalive_time=120todocker runinstead; that only affects the engine's container.🤖 Drafted with Claude Code
Set TCP keepalive idle time so idle connections survive NAT
amend! Set TCP keepalive idle time so idle connections survive NAT
Set TCP keepalive idle time so idle connections survive NAT
Set TCP keepalive idle time so idle connections survive NAT