问题

[id:48b8a8f5-1, L:/网关:37187 - R:应用/应用:应用端口] The connection observed an error, the request cannot be retried as the headers/body were sent
io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[2021-12-28T11:31:06,329] DEBUG  
 Stopping retries since predicate returned false, retry context: iteration=1 exception=io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer backoff={0ms}
[2021-12-28T11:31:06,330] ERROR  
 ==>返回错误信息
io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    |_ checkpoint ⇢ org.springframework.web.cors.reactive.CorsWebFilter [DefaultWebFilterChain]
    ...
    |_ checkpoint ⇢ HTTP POST "接口" [ExceptionHandlingWebHandler]
Stack trace:
 

分析
大体一看，是netty报的io异常，然后被reactor-netty抛出到网关返回了。
reactor-netty是SCG的底层依赖，去
https://github.com/reactor/reactor-netty
发现有同样的issue。
大体总结就是

后端应用tomcat有个keepalivetimeout，默认与connecttimeout相同，为20s。也就是说闲置20s的连接会被断开丢弃，主要跟keepalive长连接有关。
SCG网关有个连接池，存放与后端应用tomcat的通道连接，默认不释放回收，取用连接方式为FIFO先入先出（取用最早建立的连接）
以上两者，其中至少一个没有配置正确，导致SCG连接池释放闲置时间大于tomcat与网关的断开时间，就会导致浏览器向网关请求，网关获取到的连接是已经断开的连接，继续向tomcat请求，tomcat发现使用的是废弃的连接则返回reset，网关抛出异常，但后续接口不受影响。
可简单参考下图。.
解决
①配置SCG网关
spring:
  cloud:
    gateway:
      httpclient:
        response-timeout: 10s
        pool:
          type: fixed
          max-idle-time: 5000
          max-connections: 200
          acquire-timeout: 45000

②配置网关启动参数：reactor-netty取用连接的规则LIFO
-Dreactor.netty.pool.leasingStrategy=lifo

③配置tomcat的闲置断开时间
server:
  tomcat:
    connection-timeout: 10000 #根据需要
⑤配置nacos 元数据
spring: 
    cloud:
        nacos:
            discovery:
                metadata:
                    response-timeout: 10000
                    connect-timeout: 3000

④配合nginx关于keepalive的配置
upstream gateway {
#    ip_hash;
        server x.x.x.x:xx weight=1;
        server x.x.x.x:xx weight=1;
        keepalive 100;
        keepalive_requests 30000;
        keepalive_timeout 300s;
}

location /xxx{
proxy_pass http://gateway;
proxy_http_version 1.1;
proxy_set_header Connection "";
}