Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 31 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,25 @@
# fhttp
# fhttp

<!-- This note is not necessary on this repo, but I won't delete it as it should be included on the original one.
**NOTE**
This maintenance of this library has move over to [userflynet](https://github.com/useflyent/fhttp). The only use for this repository is so imports will not break.
This maintenance of this library has moved over to [useflyent](https://github.com/useflyent/fhttp). The only use for this repository is so imports will not break.

The f stands for flex. fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client. Through these changes, the http client becomes much more flexible, and when combined with transports such as [uTLS](https://github.com/refraction-networking/utls) can mitigate servers from fingerprinting requests and see that it is made by golang, making them look like they originate from a regular chrome browser.
The "f" stands for "fly" *(or "flex")*. fhttp is a fork of `net/http` that provides an array of features pertaining to the fingerprint of the golang `http` client. Through these changes, the `http` client becomes much more flexible, and when combined with transports such as [uTLS](https://github.com/refraction-networking/utls) it can mitigate fingerprinting requests, reducing the chances that a server detects they were made by a golang program, instead having them appear to originate from a regular Chrome browser.

Documentation can be contributed, otherwise, look at tests and examples. The main one should be [example_client_test.go](example_client_test.go).
-->

# Features
## Features

## Ordered Headers
Allows for pseudo header order and normal header order. Most of the code is taken from [this pr](https://go-review.googlesource.com/c/go/+/105755/).
### Ordered Headers

## Connection settings
Has Chrome-like connection settings:
```
The package allows for both pseudo header order and normal header order. Most of the code is taken from [this Pull Request](https://go-review.googlesource.com/c/go/+/105755/).

### Connection settings

fhhtp has Chrome-like connection settings, as shown below:

```text
SETTINGS_HEADER_TABLE_SIZE = 65536 (2^16)
SETTINGS_ENABLE_PUSH = 1
SETTINGS_MAX_CONCURRENT_STREAMS = 1000
Expand All @@ -23,8 +28,9 @@ SETTINGS_MAX_FRAME_SIZE = 16384 (2^14)
SETTINGS_MAX_HEADER_LIST_SIZE = 262144 (2^18)
```

Default net/http settings:
```
The default net/http settings, on the other hand, are the following:

```text
SETTINGS_HEADER_TABLE_SIZE = 4096
SETTINGS_ENABLE_PUSH = 0
SETTINGS_MAX_CONCURRENT_STREAMS = unlimited
Expand All @@ -33,34 +39,38 @@ SETTINGS_MAX_FRAME_SIZE = 16384
SETTINGS_MAX_HEADER_LIST_SIZE = 10485760
```

ENABLE_PUSH implementation was merged from [this pull request](https://go-review.googlesource.com/c/net/+/181497/)
The ENABLE_PUSH implementation was merged from [this Pull Request](https://go-review.googlesource.com/c/net/+/181497/).

### gzip, deflate, and br encoding

`gzip`, `deflate`, and `br` encoding are all supported by the package.

## gzip, deflate, br encoding
Actually supports and implements encoding `gzip, deflate, br`
### Pseudo header order

## Pseudo header order
Supports pseudo header order for http2 to mitigate fingerprinting. Read more about it [here](https://www.akamai.com/uk/en/multimedia/documents/white-paper/passive-fingerprinting-of-http2-clients-white-paper.pdf)
fhttp supports pseudo header order for http2, helping mitigate fingerprinting. You can read more about how it works [here](https://www.akamai.com/uk/en/multimedia/documents/white-paper/passive-fingerprinting-of-http2-clients-white-paper.pdf).

### Backward compatible with net/http

## Backward compatible with net/http
Although this library is an extension of `net/http`, it is also meant to be backward compatible. Replacing

```go
import (
"net/http"
"net/http"
)
```

with

```go
import (
http "github.com/useflyent/fhttp"
http "github.com/useflyent/fhttp"
)
```

SHOULD not break anything.
SHOULD not break anything.

## Credits

# Credits
Special thanks to the following people for helping me with this project.

* [cc](https://github.com/x04/) for guiding me when I first started this project and inspiring me with [cclient](https://github.com/x04/cclient)
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ go 1.16
require (
github.com/andybalholm/brotli v1.0.3
golang.org/x/net v0.0.0-20210610132358-84b48f89b13b
golang.org/x/term v0.0.0-20210503060354-a79de5458b56
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b
)
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,12 @@ golang.org/x/net v0.0.0-20210610132358-84b48f89b13b h1:k+E048sYJHyVnsr1GDrRZWQ32
golang.org/x/net v0.0.0-20210610132358-84b48f89b13b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1 h1:SrN+KX8Art/Sf4HNj6Zcz06G7VEz+7w9tdXTPOZ7+l4=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210503060354-a79de5458b56/go.mod h1:tfny5GFUkzUvx4ps4ajbZsCe5lw1metzhBm9T3x7oIY=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
7 changes: 7 additions & 0 deletions header.go
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@ var headerSorterPool = sync.Pool{
New: func() interface{} { return new(headerSorter) },
}

var mutex = &sync.RWMutex{}

// SortedKeyValues returns h's keys sorted in the returned kvs
// slice. The headerSorter used to sort is also returned, for possible
// return to headerSorterCache.
Expand All @@ -204,9 +206,11 @@ func (h Header) SortedKeyValues(exclude map[string]bool) (kvs []HeaderKeyValues,
}
kvs = hs.kvs[:0]
for k, vv := range h {
mutex.RLock()
if !exclude[k] {
kvs = append(kvs, HeaderKeyValues{k, vv})
}
mutex.RUnlock()
}
hs.kvs = kvs
sort.Sort(hs)
Expand Down Expand Up @@ -245,6 +249,7 @@ func (h Header) writeSubset(w io.Writer, exclude map[string]bool, trace *httptra

var kvs []HeaderKeyValues
var sorter *headerSorter

// Check if the HeaderOrder is defined.
if headerOrder, ok := h[HeaderOrderKey]; ok {
order := make(map[string]int)
Expand All @@ -254,7 +259,9 @@ func (h Header) writeSubset(w io.Writer, exclude map[string]bool, trace *httptra
if exclude == nil {
exclude = make(map[string]bool)
}
mutex.Lock()
exclude[HeaderOrderKey] = true
mutex.Unlock()
kvs, sorter = h.SortedKeyValuesBy(order, exclude)
} else {
kvs, sorter = h.SortedKeyValues(exclude)
Expand Down