fix(socket_mode): recreate a closed aiohttp session when reconnecting - #1966
Shubham-Padkonde wants to merge 2 commits into
Conversation
The aiohttp SocketModeClient shares one ClientSession across all connections. Once it was closed, every reconnect attempt failed with "Session is closed" and connect() retried forever. Recreate the session at the start of a connection attempt when it is closed, and report a closed session as not connected. Fixes slackapi#1922 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1966 +/- ##
==========================================
+ Coverage 84.15% 84.16% +0.01%
==========================================
Files 118 118
Lines 13566 13570 +4
==========================================
+ Hits 11416 11421 +5
+ Misses 2150 2149 -1 ☔ View full report in Codecov by Harness. |
AmyScript
left a comment
There was a problem hiding this comment.
Thanks for the fix! The recreate is safely guarded. It lives inside while not self.closed, and close() sets self.closed = True before it closes the shared session, so a normal shutdown can't trigger a spurious new session (the loop has already exited). And the not self.aiohttp_client_session.closed check in is_connected() directly addresses the production symptom in the issue which was a watchdog on is_connected() couldn't detect the dead HTTP session while the WS layer still looked connected.
WilliamBergamin
left a comment
There was a problem hiding this comment.
Thanks for picking this up 🙏
The approach looks right to me: recreating the session right before ws_connect is the smallest fix that actually ends the loop. Raising or breaking instead wouldn't be enough, because monitor_current_session catches the error and calls connect_to_new_endpoint() again every ping_interval, so the infinite loop would just move there.
A couple of suggestions to trim it down, left inline.
| # over the lifetime of your application, | ||
| # it is suggested you use a single session for the lifetime of your application | ||
| # to benefit from connection pooling. | ||
| self._loop = loop |
There was a problem hiding this comment.
I don't think we need to store the loop. connect() always runs inside the running event loop, and aiohttp.ClientSession() binds to the running loop by default. So we can drop this attribute and create the session with no arguments below.
| if self.aiohttp_client_session.closed: | ||
| # All connections share this session. Once it has been closed, | ||
| # every connection attempt fails, so it has to be recreated. | ||
| self.logger.info("The aiohttp client session is closed; creating a new one") | ||
| self.aiohttp_client_session = aiohttp.ClientSession(loop=self._loop) |
There was a problem hiding this comment.
Could this be simplified, with one extra guard?
| if self.aiohttp_client_session.closed: | |
| # All connections share this session. Once it has been closed, | |
| # every connection attempt fails, so it has to be recreated. | |
| self.logger.info("The aiohttp client session is closed; creating a new one") | |
| self.aiohttp_client_session = aiohttp.ClientSession(loop=self._loop) | |
| if self.aiohttp_client_session.closed and not self.closed: | |
| self.logger.info("The aiohttp client session is closed; creating a new one") | |
| self.aiohttp_client_session = aiohttp.ClientSession() |
There was a problem hiding this comment.
Addressed all three suggestions in 89b7542: removed the stored loop, create the replacement session with no arguments and the extra shutdown guard, and removed the HTTP-session clause from is_connected(). Added a regression for shutdown while awaiting the old WebSocket close (fails before, passes after). All 8 aiohttp tests and project lint pass. A real local WebSocket check also confirms close detection and successful reconnection. The broader run times out in the unchanged websockets backend, and Windows mypy hits the unchanged RTM signal.SIGHUP error; these limits are recorded in the PR description.
| connected: bool = ( | ||
| not self.closed | ||
| and not self.stale | ||
| and not self.aiohttp_client_session.closed |
There was a problem hiding this comment.
I think this clause can go. ws_connect never releases its connection back to the pool, so ClientSession.close() closes the WebSocket's transport too. I tested it against a local server: current_session.closed and is_connected() both flip immediately without this line, and the monitor reconnects on a fresh session.
Fixes #1922.
The aiohttp
SocketModeClientshares oneClientSessionfor all connections. Once that session was closed, every attempt inconnect()'s retry loop failed withRuntimeError: Session is closedand the client retried forever.connect()now creates a new session at the start of an attempt when the current one is closed. The loop still stops once the client itself is closed, soclose()does not trigger a new session.not self.closed, including shutdown while closing the old WebSocket.is_connected()retains its existing WebSocket-state check.The added test closes the session, runs
connect()withws_connectstubbed, and checks that the connection attempt uses a new open session; it fails before this change. The original async Socket Mode tests, ruff and mypy passed before this follow-up.🤖 Generated with Claude Code
Review follow-up validation
scripts/run_tests.sh.scripts/lint.sh --no-installpasses.scripts/run_mypy.sh --no-installencounters the existing Windows-only missingsignal.SIGHUPerror in unchanged RTM code. No full-suite or complete type-check pass is claimed for this follow-up.