🐛 [Bug]: system reachable via ICMP and via Port 8008 but screen "navy blue" with no text whatsoever

[netchron-personal]

Bug description

I start serge (docker) in an IPv6-only environment
The page loads - the background changes to the usual "navy blue" but nothing on the page actually shows up despite the website signalling me, that it has successfully loaded

Steps to reproduce

pull the latest docker container
start serge with IPv6 enabled
access the web.ui via IPv6

Environment Information

Docker version 20.10.18, build b40c2f6
Client accessing: Windows 11
Browser accessing: latest Chrome and Firefox (same behavior)

Screenshots

No response

Relevant log output

No response

Confirmations

• I'm running the latest version of the main branch.
• I checked existing issues to see if this has already been described.

[gaby]

@Smartappli Any ideas?

[netchron-personal]

Happy to provide more info if needed.

[gaby]

@JuniperChris929 Does it work from localhost?

[netchron-personal]

Yes - If I run the very same docker container on one of my Linux-Laptops it works fine.
But as soon as I run this on a server and access it from "outside" I'm greeted with this behavior.

Already checked everything firewall-related - all green. I can see the traffic hitting the container and can also see the answer-packets.

The weird thing is, that it loads the favicon (the little llama) and seems to load the page but somehow never finishes for some weird reason

[netchron-personal]

Apologies - I tested too quickly ;)
The Laptops run Dual-Stack - so they serve the page over IPv4.
If I "force" IPv6 they also show this weird behavior.

[netchron-personal]

chs@llm-docker:~$ docker logs 4c755f694744

[2024-02-21 23:36:02 +0000] [26] [ERROR] Error in ASGI Framework

• Exception Group Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 109, in call
  | await response(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 270, in call
  | async with anyio.create_task_group() as task_group:
  | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in aexit
  | raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 273, in wrap
  | await func()
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 134, in stream_response
  | return await super().stream_response(send)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 262, in stream_response
  | async for chunk in self.body_iterator:
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 98, in body_stream
  | raise app_exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro
  | await self.app(scope, receive_or_disconnect, send_no_error)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
  | await self.app(scope, receive, sender)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
  | raise e
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in call
  | await route.handle(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 443, in handle
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 276, in call
  | await super().call(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in call
  | await self.middleware_stack(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in call
  | await self.app(scope, receive, _send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
  | await self.app(scope, receive, sender)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
  | raise e
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in call
  | await route.handle(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
  | response = await func(request)
  | ^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 237, in app
  | raw_response = await run_endpoint_function(
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
  | return await dependant.call(**values)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/serge/routers/chat.py", line 76, in get_all_chats
  | [await get_specific_chat(id.decode()) for id in ids],
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/serge/routers/chat.py", line 76, in
  | [await get_specific_chat(id.decode()) for id in ids],
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/serge/routers/chat.py", line 106, in get_specific_chat
  | chat = Chat.parse_raw(chat_raw)
  | ^^^^^^^^^^^^^^^^^^^^^^^^
  | File "pydantic/main.py", line 549, in pydantic.main.BaseModel.parse_raw
  | File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  | File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
  | pydantic.error_wrappers.ValidationError: 1 validation error for Chat
  | params -> n_gpu_layers
  | field required (type=value_error.missing)
  +------------------------------------

During handling of the above exception, another exception occurred:

• Exception Group Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/hypercorn/asyncio/task_group.py", line 27, in _handle
  | await app(scope, receive, send, sync_spawn, call_soon)
  | File "/usr/local/lib/python3.11/site-packages/hypercorn/app_wrappers.py", line 34, in call
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 276, in call
  | await super().call(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in call
  | await self.middleware_stack(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in call
  | await self.app(scope, receive, _send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 84, in call
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 106, in call
  | async with anyio.create_task_group() as task_group:
  | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in aexit
  | raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 277, in call
  | await wrap(partial(self.listen_for_disconnect, receive))
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 273, in wrap
  | await func()
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 250, in listen_for_disconnect
  | message = await receive()
  | ^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/asyncio/queues.py", line 158, in get
  | await getter
  | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fa818d69910
  |
  | During handling of the above exception, another exception occurred:
  |
  | Exception Group Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 109, in call
  | await response(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 270, in call
  | async with anyio.create_task_group() as task_group:
  | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in aexit
  | raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 273, in wrap
  | await func()
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 134, in stream_response
  | return await super().stream_response(send)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 262, in stream_response
  | async for chunk in self.body_iterator:
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 98, in body_stream
  | raise app_exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro
  | await self.app(scope, receive_or_disconnect, send_no_error)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
  | await self.app(scope, receive, sender)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
  | raise e
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in call
  | await route.handle(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 443, in handle
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 276, in call
  | await super().call(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in call
  | await self.middleware_stack(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in call
  | await self.app(scope, receive, _send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
  | raise exc
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
  | await self.app(scope, receive, sender)
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
  | raise e
  | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in call
  | await route.handle(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
  | await self.app(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
  | response = await func(request)
  | ^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 237, in app
  | raw_response = await run_endpoint_function(
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
  | return await dependant.call(**values)
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/serge/routers/chat.py", line 76, in get_all_chats
  | [await get_specific_chat(id.decode()) for id in ids],
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/serge/routers/chat.py", line 76, in
  | [await get_specific_chat(id.decode()) for id in ids],
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | File "/usr/local/lib/python3.11/site-packages/serge/routers/chat.py", line 106, in get_specific_chat
  | chat = Chat.parse_raw(chat_raw)
  | ^^^^^^^^^^^^^^^^^^^^^^^^
  | File "pydantic/main.py", line 549, in pydantic.main.BaseModel.parse_raw
  | File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  | File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
  | pydantic.error_wrappers.ValidationError: 1 validation error for Chat
  | params -> n_gpu_layers
  | field required (type=value_error.missing)
  +------------------------------------

[netchron-personal]

Do you need anything from my side?

[netchron-personal]

just saw that you implemented some fixes for IPv6 in the last commit(s) - will you release a new Version soon so that I can test of this fixes it? If I pull the "latest" tag from docker - does that already include the latest commits or is this just the latest released version?

[netchron-personal]

any news regarding this fix? Can we expect a new version soon? :)

[netchron-personal]

Any updates when a new version can be expected? It's been 4 months now and you probably already fixed a lot of issues

[gaby]

@JuniperChris929 Does this happen with ghcr.io/serge-char/serge:main ?

[netchron-personal]

Yes - same behavior (once I say IPv4 = false and IPv6 = true in serge.env

[netchron-personal]

[2024-06-28 20:52:19 +0000] [20] [ERROR] Error in ASGI Framework

• Exception Group Traceback (most recent call last):
  | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 109, in call
  | await response(scope, receive, send)
  | File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 270, in call
  | async with anyio.create_task_group() as task_group:
  | File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
  | raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------

[gaby]

@Smartappli Any idea?