Spring Cloud Gateway max HTTP header size
I've already written about my experiences using Spring Cloud Gateway. While the framework continues to provide a great set of features, it does have its quirks. This is a story about one of them.
Use-case
Like a lot of stories, this one begins with a feature request. A few days ago I received a request from a client who wished to send large amounts of data through query parameters. Servers generally have a default limit on the maximum HTTP request size and will reject offending requests with HTTP status code 413 or 414. This behavior is configurable, and Spring Boot, which Cloud Gateway builds upon, provides a nice configuration abstraction that works with different servers.
This should be easy; 1 story point.
famous last words
1st attempt
Spring Boot's common application properties include a property for just this situation: server.max-http-header-size. At first, I thought I'd solve the tasks with a single config line in my app's application.yaml. Luckily I also added a test to cover this case. In my tests I use WireMock to simulate the backend service and then send requests through the gateway application. Test was rather straightforward:
@Test
void shouldSupportLongQueryParameters() {
// given
var givenQueryParamLength = 10_000;
var givenQueryParam = new String(new char[givenQueryParamLength]).replaceAll("\0", "a");
stubFor(get(urlEqualTo("/foo/bars?q=" + givenQueryParam))
.willReturn(aResponse().withStatus(200))
);
// when
var actual = client.get().uri("/foo/bars?q=" + givenQueryParam).exchange();
// then
actual.expectStatus().isOk();
}
To my surprise, running it resulted in an assertion failure:
Status expected:<200 OK> but was:<413 PAYLOAD_TOO_LARGE>
2nd attempt
Ok, I guess the common application property doesn't work. No problem. Spring Cloud Gateway is using Reactor Netty web server under the hood. So the next thing I looked at was that project's documentation. And indeed, as per documentation, Netty has 2 configuration parameters of note:
- The maximum length of the initial HTTP request line
- The maximum length of all headers
That explained why setting the max-http-header-size config property did not solve the issue: Netty requires a separate parameter for the initial line length, which query parameters are a part of. At this point I registered a slightly customized instance of NettyReactiveWebServerFactory as a bean in my context. In it I set both max header size and max initial line length:
@Bean
public NettyReactiveWebServerFactory nettyFactory(ServerProperties serverProperties) {
var maxInBytes = (int) serverProperties.getMaxHttpHeaderSize().toBytes();
var factory = new NettyReactiveWebServerFactory();
factory.addServerCustomizers(
server -> server.httpRequestDecoder(
reqDecoder -> reqDecoder
.maxInitialLineLength(maxInBytes)
.maxHeaderSize(maxInBytes)
)
);
return factory;
}
That got me absolutely nowhere:
Status expected:<200 OK> but was:<413 PAYLOAD_TOO_LARGE>
3rd attempt
From this failure I realized I'm still missing something crucial about the way in which Spring Boot sets up the Netty server. So I sprinkled a few breakpoints around the code and started debugging. The first thing I noticed was that my server factory's addServerCustomizers method was getting invoked a second time from within Spring code, specifically from NettyWebServerFactoryCustomizer.
Turns out I didn't have to instantiate the entire server factory bean. There's a factory customization mechanism and all I needed to do was provide my own implementation of the customizer interface. The one implementation that Spring provides sets the Netty's max header size to the value of max-http-header-size application property. However, this did not explain why the value of max initial line length I was setting in my factory bean was getting ignored.
To understand this I had to look into Reactor's HttpServer, specifically the code that factory customizers use to set max header and initial line values. According to the implementation of the HttpServer::httpRequestDecoder method each attempt to customize those values causes server to create a brand new spec object. This explained how Spring's factory customizer, which only sets max header size, managed to override my initial line config. What I needed to do was provide a factory customizer with lower order of execution, so that my values don't get overridden. I created my own WebServerFactoryCustomizer, and set its order to Ordered.LOWEST_PRECEDENCE. Implementation was the same as before: I set both max header size and initial line length to the same value.
I rerun the test and got:
Status expected:<200 OK> but was:<414 URI_TOO_LONG>
That was... different?
4th attempt
I've already established that Netty will return status code 413 if incoming request goes over max initial line setting. So this latest issue must have been somewhere else. Going through my test output log I found this mystery line:
w.org.eclipse.jetty.http.HttpParser : URI is too large >8192
Where did Jetty come from? Well, there were actually 2 servers in the test: Spring Cloud Gateway app running on Netty and WireMock running on Jetty. Which meant my last attempt worked, I just needed to fix the test! Up until then I relied on Spring to auto-configure WireMock for me. However, I now needed to customize the max URL size for it. Luckily, this can be done easily by providing an instance of an Options bean:
@Bean
public Options wireMockOptions(@Value("${wiremock.server.port}") int port) {
return WireMockConfiguration.options()
.port(port)
.jettyHeaderBufferSize(16_384);
}
Jetty header buffer size was the only property that WireMock needed in order to configure larger max URL size. With this change the test finally passed, and the feature was complete!
Example
I've created a demo project on GitHub that illustrates this implementation. I marked each of my attempts with a git tag, so you can follow them step by step. Feel free to check out the project and play around with it. If you find a better solution, please let me know about it! You can reach me at: