Skip to content

fix(socket_mode): recreate a closed aiohttp session when reconnecting - #1966

Open
Shubham-Padkonde wants to merge 2 commits into
slackapi:mainfrom
Shubham-Padkonde:fix/socket-mode-closed-aiohttp-session
Open

Shubham-Padkonde wants to merge 2 commits into
slackapi:mainfrom
Shubham-Padkonde:fix/socket-mode-closed-aiohttp-session

Conversation

@Shubham-Padkonde

@Shubham-Padkonde Shubham-Padkonde commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Fixes #1922.

The aiohttp SocketModeClient shares one ClientSession for all connections. Once that session was closed, every attempt in connect()'s retry loop failed with RuntimeError: Session is closed and the client retried forever.

  • connect() now creates a new session at the start of an attempt when the current one is closed. The loop still stops once the client itself is closed, so close() does not trigger a new session.
  • Session recreation uses the running loop and is guarded by not self.closed, including shutdown while closing the old WebSocket. is_connected() retains its existing WebSocket-state check.

The added test closes the session, runs connect() with ws_connect stubbed, and checks that the connection attempt uses a new open session; it fails before this change. The original async Socket Mode tests, ruff and mypy passed before this follow-up.

🤖 Generated with Claude Code

Review follow-up validation

  • New shutdown-during-reconnect regression fails before the guard and passes after it. All 8 aiohttp tests pass via scripts/run_tests.sh.
  • A real local aiohttp WebSocket check confirms close detection and reconnection with a fresh HTTP session.
  • scripts/lint.sh --no-install passes.
  • Broader Socket Mode run passes 13 tests before timing out in the unchanged websockets backend after 180 seconds.
  • scripts/run_mypy.sh --no-install encounters the existing Windows-only missing signal.SIGHUP error in unchanged RTM code. No full-suite or complete type-check pass is claimed for this follow-up.

The aiohttp SocketModeClient shares one ClientSession across all
connections. Once it was closed, every reconnect attempt failed with
"Session is closed" and connect() retried forever. Recreate the session
at the start of a connection attempt when it is closed, and report a
closed session as not connected.

Fixes slackapi#1922

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Shubham-Padkonde
Shubham-Padkonde requested a review from a team as a code owner September 19, 2026 04:15
@AmyScript
AmyScript self-requested a review September 22, 2026 15:09
@codecov

codecov Bot commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.16%. Comparing base (b9f4666) to head (4a754cd).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1966      +/-   ##
==========================================
+ Coverage   84.15%   84.16%   +0.01%     
==========================================
  Files         118      118              
  Lines       13566    13570       +4     
==========================================
+ Hits        11416    11421       +5     
+ Misses       2150     2149       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@AmyScript AmyScript left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! The recreate is safely guarded. It lives inside while not self.closed, and close() sets self.closed = True before it closes the shared session, so a normal shutdown can't trigger a spurious new session (the loop has already exited). And the not self.aiohttp_client_session.closed check in is_connected() directly addresses the production symptom in the issue which was a watchdog on is_connected() couldn't detect the dead HTTP session while the WS layer still looked connected.

@WilliamBergamin WilliamBergamin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up 🙏

The approach looks right to me: recreating the session right before ws_connect is the smallest fix that actually ends the loop. Raising or breaking instead wouldn't be enough, because monitor_current_session catches the error and calls connect_to_new_endpoint() again every ping_interval, so the infinite loop would just move there.

A couple of suggestions to trim it down, left inline.

# over the lifetime of your application,
# it is suggested you use a single session for the lifetime of your application
# to benefit from connection pooling.
self._loop = loop

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to store the loop. connect() always runs inside the running event loop, and aiohttp.ClientSession() binds to the running loop by default. So we can drop this attribute and create the session with no arguments below.

Comment on lines +374 to +378
if self.aiohttp_client_session.closed:
# All connections share this session. Once it has been closed,
# every connection attempt fails, so it has to be recreated.
self.logger.info("The aiohttp client session is closed; creating a new one")
self.aiohttp_client_session = aiohttp.ClientSession(loop=self._loop)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be simplified, with one extra guard?

Suggested change
if self.aiohttp_client_session.closed:
# All connections share this session. Once it has been closed,
# every connection attempt fails, so it has to be recreated.
self.logger.info("The aiohttp client session is closed; creating a new one")
self.aiohttp_client_session = aiohttp.ClientSession(loop=self._loop)
if self.aiohttp_client_session.closed and not self.closed:
self.logger.info("The aiohttp client session is closed; creating a new one")
self.aiohttp_client_session = aiohttp.ClientSession()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed all three suggestions in 89b7542: removed the stored loop, create the replacement session with no arguments and the extra shutdown guard, and removed the HTTP-session clause from is_connected(). Added a regression for shutdown while awaiting the old WebSocket close (fails before, passes after). All 8 aiohttp tests and project lint pass. A real local WebSocket check also confirms close detection and successful reconnection. The broader run times out in the unchanged websockets backend, and Windows mypy hits the unchanged RTM signal.SIGHUP error; these limits are recorded in the PR description.

connected: bool = (
not self.closed
and not self.stale
and not self.aiohttp_client_session.closed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this clause can go. ws_connect never releases its connection back to the pool, so ClientSession.close() closes the WebSocket's transport too. I tested it against a local server: current_session.closed and is_connected() both flip immediately without this line, and the monitor reconnects on a fresh session.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SocketModeClient.connect() retries forever against a permanently closed aiohttp ClientSession

3 participants