Add support for ~ expansion for DB filepath

* update readme

* update gitignore (test binary, built with go test -c)

* update readme

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+# These are supported funding model platforms
+
+patreon: dstotijn

.gitignore

@@ -1,4 +1,5 @@
 **/rice-box.go
 dist
 hetty
-hetty.bolt
+hetty.bolt
+*.test

Dockerfile

@@ -25,4 +25,4 @@ COPY --from=node-builder /app/dist admin
 
 ENTRYPOINT ["./hetty", "-adminPath=./admin"]
 
-EXPOSE 80
+EXPOSE 8080

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 David Stotijn
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,129 @@
+<img src="https://i.imgur.com/AT71SBq.png" width="346" />
+
+> Hetty is an HTTP toolkit for security research. It aims to become an open source
+> alternative to commercial software like Burp Suite Pro, with powerful features
+> tailored to the needs of the infosec and bug bounty community.
+
+<img src="https://i.imgur.com/ZZ6o83X.png">
+
+## Features/to do
+
+- [x] HTTP man-in-the-middle (MITM) proxy and GraphQL server.
+- [x] Web interface (Next.js) with proxy log viewer.
+- [ ] Add scope support to the proxy.
+- [ ] Full text search (with regex) in proxy log viewer.
+- [ ] Project management.
+- [ ] Sender module for sending manual HTTP requests, either from scratch or based
+      off requests from the proxy log.
+- [ ] Attacker module for automated sending of HTTP requests. Leverage the concurrency
+      features of Go and its `net/http` package to make it blazingly fast.
+
+## Installation
+
+Hetty is packaged on GitHub as a single binary, with the web interface resources
+embedded.
+
+👉 You can find downloads for Linux, macOS and Windows on the [releases page](https://github.com/dstotijn/hetty/releases).
+
+### Alternatives:
+
+**Build from source**
+
+```
+$ GO111MODULE=auto go get -u -v github.com/dstotijn/hetty/cmd/hetty
+```
+
+Then export the Next.js frontend app:
+
+```
+$ cd admin
+$ yarn install
+$ yarn export
+```
+
+This will ensure a folder `./admin/dist` exists.
+Then, you can bundle the frontend app using `rice`.
+The easiest way to do this is via a supplied `Makefile` command in the root of
+the project:
+
+```
+make build
+```
+
+**Docker**
+
+Alternatively, you can run Hetty via Docker. See: [`dstotijn/hetty`](https://hub.docker.com/r/dstotijn/hetty)
+on Docker Hub.
+
+```
+$ docker run \
+-v $HOME/.hetty/hetty_key.pem:/root/.hetty/hetty_key.pem \
+-v $HOME/.hetty/hetty_cert.pem:/root/.hetty/hetty_cert.pem \
+-v $HOME/.hetty/hetty.bolt:/root/.hetty/hetty.bolt \
+-p 127.0.0.1:8080:8080 \
+dstotijn/hetty
+```
+
+## Usage
+
+Hetty is packaged as a single binary, with the web interface resources embedded.
+When the program is run, it listens by default on `:8080` and is accessible via
+http://localhost:8080. Depending on incoming HTTP requests, it either acts as a
+MITM proxy, or it serves the GraphQL API and web interface (Next.js).
+
+```
+$ Usage of ./hetty:
+  -addr string
+        TCP address to listen on, in the form "host:port" (default ":80")
+  -adminPath string
+        File path to admin build
+  -cert string
+        CA certificate filepath. Creates a new CA certificate is file doesn't exist (default "~/.hetty/hetty_cert.pem")
+  -db string
+        Database file path (default "hetty.db")
+  -key string
+        CA private key filepath. Creates a new CA private key if file doesn't exist (default "~/.hetty/hetty_key.pem")
+```
+
+⚠️ _Todo: Write instructions for installing CA certificate in local CA store, and_
+_configuring Hetty to be used as a proxy server._
+
+## Vision and roadmap
+
+The project has just gotten underway, and as such I haven’t had time yet to do a
+write-up on its mission and roadmap. A short summary/braindump:
+
+- Fast core/engine, built with Go, with a minimal memory footprint.
+- GraphQL server to interact with the backend.
+- Easy to use web interface, built with Next.js and Material UI.
+- Extensibility is top of mind. All modules are written as Go packages, to
+  be used by the main `hetty` program, but also usable as libraries for other software.
+  Aside from the GraphQL server, it should (eventually) be possible to also use
+  it as a CLI tool.
+- Pluggable architecture for the MITM proxy and future modules, making it
+  possible for hook into the core engine.
+- I’ve chosen [Cayley](https://cayley.io/) as the graph database (backed by
+  BoltDB storage on disk) for now (not sure if it will work in the long run).
+  The benefit is that Cayley (also written in Go)
+  is embedded as a library. Because of this, the complete application is self contained
+  in a single running binary.
+- Talk to the community, and focus on the features that the majority.
+  Less features means less code to maintain.
+
+## Status
+
+The project is currently under active development. Please star/follow and check
+back soon. 🤗
+
+## Acknowledgements
+
+Thanks to the [Hacker101 community on Discord](https://discordapp.com/channels/514337135491416065)
+for all the encouragement to actually start building this thing!
+
+## License
+
+[MIT](LICENSE)
+
+---
+
+© 2020 David Stotijn — [Twitter](https://twitter.com/dstotijn), [Email](mailto:dstotijn@gmail.com)

cmd/hetty/main.go

@@ -2,7 +2,6 @@ package main
 
 import (
 	"crypto/tls"
-	"crypto/x509"
 	"flag"
 	"log"
 	"net"
@@ -19,6 +18,7 @@ import (
 	"github.com/99designs/gqlgen/graphql/handler"
 	"github.com/99designs/gqlgen/graphql/playground"
 	"github.com/gorilla/mux"
+	"github.com/mitchellh/go-homedir"
 )
 
 var (
@@ -30,21 +30,32 @@ var (
 )
 
 func main() {
-	flag.StringVar(&caCertFile, "cert", "", "CA certificate file path")
+	flag.StringVar(&caCertFile, "cert", "~/.hetty/hetty_cert.pem", "CA certificate filepath. Creates a new CA certificate is file doesn't exist")
-	flag.StringVar(&caKeyFile, "key", "", "CA private key file path")
+	flag.StringVar(&caKeyFile, "key", "~/.hetty/hetty_key.pem", "CA private key filepath. Creates a new CA private key if file doesn't exist")
-	flag.StringVar(&dbFile, "db", "hetty.db", "Database file path")
+	flag.StringVar(&dbFile, "db", "~/.hetty/hetty.bolt", "Database file path")
-	flag.StringVar(&addr, "addr", ":80", "TCP address to listen on, in the form \"host:port\"")
+	flag.StringVar(&addr, "addr", ":8080", "TCP address to listen on, in the form \"host:port\"")
 	flag.StringVar(&adminPath, "adminPath", "", "File path to admin build")
 	flag.Parse()
 
-	tlsCA, err := tls.LoadX509KeyPair(caCertFile, caKeyFile)
+	// Expand `~` in filepaths.
+	caCertFile, err := homedir.Expand(caCertFile)
 	if err != nil {
-		log.Fatalf("[FATAL] Could not load CA key pair: %v", err)
+		log.Fatalf("[FATAL] Could not parse CA certificate filepath: %v", err)
+	}
+	caKeyFile, err := homedir.Expand(caKeyFile)
+	if err != nil {
+		log.Fatalf("[FATAL] Could not parse CA private key filepath: %v", err)
+	}
+	dbFile, err := homedir.Expand(dbFile)
+	if err != nil {
+		log.Fatalf("[FATAL] Could not parse CA private key filepath: %v", err)
 	}
 
-	caCert, err := x509.ParseCertificate(tlsCA.Certificate[0])
+	// Load existing CA certificate and key from disk, or generate and write
+	// to disk if no files exist yet.
+	caCert, caKey, err := proxy.LoadOrCreateCA(caKeyFile, caCertFile)
 	if err != nil {
-		log.Fatalf("[FATAL] Could not parse CA: %v", err)
+		log.Fatalf("[FATAL] Could not create/load CA key pair: %v", err)
 	}
 
 	db, err := cayley.NewDatabase(dbFile)
@@ -55,7 +66,7 @@ func main() {
 
 	reqLogService := reqlog.NewService(db)
 
-	p, err := proxy.NewProxy(caCert, tlsCA.PrivateKey)
+	p, err := proxy.NewProxy(caCert, caKey)
 	if err != nil {
 		log.Fatalf("[FATAL] Could not create Proxy: %v", err)
 	}

go.mod

@@ -10,6 +10,7 @@ require (
 	github.com/google/uuid v1.1.2
 	github.com/gorilla/mux v1.7.4
 	github.com/hidal-go/hidalgo v0.0.0-20190814174001-42e03f3b5eaa
+	github.com/mitchellh/go-homedir v1.1.0
 	github.com/vektah/gqlparser/v2 v2.0.1
 	golang.org/x/tools v0.0.0-20200925191224-5d1fdd8fa346 // indirect
 )

go.sum

@@ -52,6 +52,7 @@ github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8Nz
 github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
 github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
 github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
+github.com/cpuguy83/go-md2man v1.0.10 h1:BSKMNlYxDvnunlTymqtgONjNnaRV1sTpcovwwjF22jk=
 github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
 github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d h1:U+s90UTSYgptZMwQh2aRr3LuazLJIa+Pg3Kc1ylSYVY=
 github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
@@ -210,6 +211,7 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky
 github.com/mattn/go-sqlite3 v1.10.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
 github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
 github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
+github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
 github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
 github.com/mitchellh/mapstructure v0.0.0-20180203102830-a4e142e9c047 h1:zCoDWFD5nrJJVjbXiDZcVhOBSzKn3o9LgRLLMRNuru8=
 github.com/mitchellh/mapstructure v0.0.0-20180203102830-a4e142e9c047/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
@@ -268,6 +270,7 @@ github.com/rogpeppe/go-internal v1.4.0/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTE
 github.com/rogpeppe/go-internal v1.5.0 h1:Usqs0/lDK/NqTkvrmKSwA/3XkZAs7ZAW/eLeQ2MVBTw=
 github.com/rogpeppe/go-internal v1.5.0/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
 github.com/rs/cors v1.6.0/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU=
+github.com/russross/blackfriday v1.5.2 h1:HyvC0ARfnZBqnXwABFeSZHpKvJHJJfPz81GNueLj0oo=
 github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
 github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
 github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=

modd.conf

@@ -1,12 +1,3 @@
-@cert = $HOME/.ssh/hetty_cert.pem
-@key = $HOME/.ssh/hetty_key.pem
-@db = hetty.bolt
-@addr = :8080
-
 **/*.go {
-    daemon +sigterm: go run ./cmd/hetty \
+    daemon +sigterm: go run ./cmd/hetty
-        -cert=@cert \
-        -key=@key \
-        -db=@db \
-        -addr=@addr
 }

pkg/proxy/cert.go

@@ -9,9 +9,13 @@ import (
 	"crypto/tls"
 	"crypto/x509"
 	"crypto/x509/pkix"
+	"encoding/pem"
 	"errors"
+	"fmt"
 	"math/big"
 	"net"
+	"os"
+	"path/filepath"
 	"time"
 )
 
@@ -56,6 +60,70 @@ func NewCertConfig(ca *x509.Certificate, caPrivKey crypto.PrivateKey) (*CertConf
 	}, nil
 }
 
+// LoadOrCreateCA loads an existing CA key pair from disk, or creates
+// a new keypair and saves to disk if certificate or key files don't exist.
+func LoadOrCreateCA(caKeyFile, caCertFile string) (*x509.Certificate, *rsa.PrivateKey, error) {
+	tlsCA, err := tls.LoadX509KeyPair(caCertFile, caKeyFile)
+	if err == nil {
+		caCert, err := x509.ParseCertificate(tlsCA.Certificate[0])
+		if err != nil {
+			return nil, nil, fmt.Errorf("proxy: could not parse CA: %v", err)
+		}
+		caKey, ok := tlsCA.PrivateKey.(*rsa.PrivateKey)
+		if !ok {
+			return nil, nil, errors.New("proxy: private key is not RSA")
+		}
+		return caCert, caKey, nil
+	}
+	if !os.IsNotExist(err) {
+		return nil, nil, fmt.Errorf("proxy: could not load CA key pair: %v", err)
+	}
+
+	// Create directories for files if they don't exist yet.
+	keyDir, _ := filepath.Split(caKeyFile)
+	if keyDir != "" {
+		if _, err := os.Stat(keyDir); os.IsNotExist(err) {
+			os.Mkdir(keyDir, 0755)
+		}
+	}
+	keyDir, _ = filepath.Split(caCertFile)
+	if keyDir != "" {
+		if _, err := os.Stat("keyDir"); os.IsNotExist(err) {
+			os.Mkdir(keyDir, 0755)
+		}
+	}
+
+	// Create new CA keypair.
+	caCert, caKey, err := NewCA("Hetty", "Hetty CA", time.Duration(365*24*time.Hour))
+	if err != nil {
+		return nil, nil, fmt.Errorf("proxy: could not generate new CA keypair: %v", err)
+	}
+
+	// Open CA certificate and key files for writing.
+	certOut, err := os.Create(caCertFile)
+	if err != nil {
+		return nil, nil, fmt.Errorf("proxy: could not open cert file for writing: %v", err)
+	}
+	keyOut, err := os.OpenFile(caKeyFile, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
+	if err != nil {
+		return nil, nil, fmt.Errorf("proxy: could not open key file for writing: %v", err)
+	}
+
+	// Write PEM blocks to CA certificate and key files.
+	if err := pem.Encode(certOut, &pem.Block{Type: "CERTIFICATE", Bytes: caCert.Raw}); err != nil {
+		return nil, nil, fmt.Errorf("proxy: could not write CA certificate to disk: %v", err)
+	}
+	privBytes, err := x509.MarshalPKCS8PrivateKey(caKey)
+	if err != nil {
+		return nil, nil, fmt.Errorf("proxy: could not convert private key to DER format: %v", err)
+	}
+	if err := pem.Encode(keyOut, &pem.Block{Type: "PRIVATE KEY", Bytes: privBytes}); err != nil {
+		return nil, nil, fmt.Errorf("proxy: could not write CA key to disk: %v", err)
+	}
+
+	return caCert, caKey, nil
+}
+
 // NewCA creates a new CA certificate and associated private key.
 func NewCA(name, organization string, validity time.Duration) (*x509.Certificate, *rsa.PrivateKey, error) {
 	priv, err := rsa.GenerateKey(rand.Reader, 2048)
@@ -91,8 +159,8 @@ func NewCA(name, organization string, validity time.Duration) (*x509.Certificate
 		KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
 		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
 		BasicConstraintsValid: true,
-		NotBefore:             time.Now().Add(-24 * time.Hour),
+		NotBefore:             time.Now(),
-		NotAfter:              time.Now().Add(24 * time.Hour),
+		NotAfter:              time.Now().Add(validity),
 		DNSNames:              []string{name},
 		IsCA:                  true,
 	}