Add proteus.Builder and methods to generate individual functions and …

…ad-hoc queries (#42)

* add support for building individual functions. add support for running queries without building a function. minor error message fixes.

* use a map[string]interface{} to combine the names and params into a single input parameter for Exec and Query. Validate that we can use map[string]interface{} as an input or output parameter.

* add unit test coverage (99% on new code). Fix more error messages.

* update README

README.md

@@ -8,6 +8,15 @@ A simple tool for generating an application's data access layer.
 
 ## Purpose
 
+Proteus makes your SQL queries type-safe and prevents SQL injection attacks. It processes structs with struct tags on function fields to generate 
+Go functions at runtime. These functions map input parameters to SQL query parameters and optionally map the output parameters to the output of your
+SQL queries.
+
+In addition to being type-safe, Proteus also prevents SQL injection by generating prepared statements from your SQL queries. Even dynamic `in` clauses 
+are converted into injection-proof prepared statements.
+
+Proteus is _not_ an ORM; it does not generate SQL. It just automates away the boring parts of interacting with databases in Go.
+
 ## Quick Start
 1. Define a struct that contains function fields and tags to indicate the query and the parameter names:
 
@@ -219,6 +228,101 @@ as the second parameter.
 
 Note that if you are using the context variants, you cannot use the `proteus.Wrap` function; this is OK, as Wrap is now a no-op and is considered deprecated.
 
+## Generating function variables
+
+Some people don't want to use structs and struct tags to implement their SQL mapping layer. Starting with version 0.11, Proteus can also generate functions that aren't fields in a struct.
+
+First, create an instance of a `proteus.Builder`. The factory function takes a `proteus.Adapter` and zero or more `proteus.QueryMapper` instances:
+
+```go
+    b := NewBuilder(Postgres)
+```
+
+Next, declare a function variable with the signature you want. The parameters for the function variables follow the same rules as the function fields: 
+
+```go
+    var f func(c context.Context, e ContextExecutor, name string, age int) (int64, error)
+    var g func(c context.Context, q ContextQuerier, id int) (*Person, error)
+```
+
+Then call the `BuildFunction` method on your `proteus.Builder` instance, passing in a pointer to your function variable, the SQL query (or a query mapper reference),
+and the parameter names as a string slice:
+
+```go
+    err := b.BuildFunction(ctx, &f, "INSERT INTO PERSON(name, age) VALUES(:name:, :age:)", []string{"name", "age"})
+    if err != nil {
+        t.Fatalf("build function failed: %v", err)
+    }
+
+    err = b.BuildFunction(ctx, &g, "SELECT * FROM PERSON WHERE id = :id:", []string{"id"})
+    if err != nil {
+        t.Fatalf("build function 2 failed: %v", err)
+    }
+```
+
+Finally, call your functions, to run your SQL queries:
+
+```go
+    db := setupDbPostgres()
+    defer db.Close()
+    ctx := context.Background()
+
+    rows, err := f(ctx, db, "Fred", 20)
+    if err != nil {
+        t.Fatalf("create failed: %v", err)
+    }
+    fmt.Println(rows) // prints 1
+
+    p, err := g(ctx, db, 1)
+    if err != nil {
+        t.Fatalf("get failed: %v", err)
+    }
+    fmt.Println(p) // prints {1, Fred, 20}
+```
+
+## Ad-hoc database queries
+
+While Proteus is focused on type safety, sometimes you just want to run a query without associating it with a function. 
+Starting with version 0.11, Proteus allows you to run ad-hoc database queries.
+
+First, create an instance of a `proteus.Builder`. The factory function takes a `proteus.Adapter` and zero or more `proteus.QueryMapper` instances:
+
+```go
+    b := NewBuilder(Postgres)
+```
+
+Next, run your query by passing it to the `Exec` or `Query` methods on `proteus.Builder`. 
+
+`Exec` expects a `context.Context`, a `proteus.ContextExecutor`, the query, and a 
+map of `string` to `interface{}`, where the keys are the parameter names and the values are the parameter values. It returns an int64 with the number of rows modified and
+and error. 
+
+`Query` expected a `context.Context`, a `proteus.ContextQuerier`, the query, 
+a map of `string` to `interface{}`, where the keys are the parameter names and the values are the parameter values, and a pointer to the value that
+should be populated by the query. The method returns an error.
+
+```go
+    db := setupDbPostgres()
+    defer db.Close()
+    ctx := context.Background()
+
+    rows, err := b.Exec(c, db, "INSERT INTO PERSON(name, age) VALUES(:name:, :age:)", map[string]interface{}{"name": "Fred", "age": 20})
+    if err != nil {
+        t.Fatalf("create failed: %v", err)
+    } 
+    fmt.Println(rows) // prints 1
+
+    var p *Person
+    err = b.Query(c, db, "SELECT * FROM PERSON WHERE id = :id:", map[string]interface{}{"id": 1}, &p)
+    if err != nil {
+        t.Fatalf("get failed: %v", err)
+    }
+    fmt.Println(*p) // prints {1, Fred, 20}
+```
+
+Ad-hoc queries support all of the functionality of Proteus except for type safety. You can reference queries in `proteus.QueryMapper` instances, build out dynamic
+`in` clauses, extract values from `struct` instances, and map to a struct with `prof` tags on its fields.  
+
 ## Valid function signatures
 
 ## API
@@ -352,6 +456,10 @@ Feel free to use this logger within your own code. If this logger proves to be u
 
 There are more interesting features coming to Proteus. They are (in likely order of implementation):
 
+- Build `in` clauses using a field from a slice of struct or map
+
+- Generate batch `values` clauses using a slice of struct or map
+
 - more expansive performance measurement support and per-request logging control
 
 - go generate tool to create a wrapper struct and interface for a Proteus DAO

builder.go

@@ -58,7 +58,11 @@ var (
 	valueType = reflect.TypeOf((*driver.Valuer)(nil)).Elem()
 )
 
-func buildFixedQueryAndParamOrder(c context.Context, query string, nameOrderMap map[string]int, funcType reflect.Type, pa ParamAdapter) (queryHolder, []paramInfo, error) {
+type posType interface {
+	In(i int) reflect.Type
+}
+
+func buildFixedQueryAndParamOrder(c context.Context, query string, nameOrderMap map[string]int, funcType posType, pa ParamAdapter) (queryHolder, []paramInfo, error) {
 	var out bytes.Buffer
 
 	var paramOrder []paramInfo

go.mod

@@ -3,6 +3,7 @@ module github.com/jonbodner/proteus
 go 1.12
 
 require (
+	github.com/google/go-cmp v0.3.0
 	github.com/jonbodner/dbtimer v0.0.0-20170410163237-7002f3758ae1
 	github.com/jonbodner/multierr v0.0.0-20181226035711-ce28e9ccaa7d
 	github.com/lib/pq v1.1.1

go.sum

@@ -1,6 +1,8 @@
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
+github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/jonbodner/dbtimer v0.0.0-20170410163237-7002f3758ae1 h1:mgFL7UFb88FOlSVgVoIRGJ4yKlkfp8KcXHqy7no+lEU=
 github.com/jonbodner/dbtimer v0.0.0-20170410163237-7002f3758ae1/go.mod h1:PjOlFbeJKs+4b2CvuN9FFF8Ed8cZ6FHWPb5tLK2QKOM=
 github.com/jonbodner/multierr v0.0.0-20181226035711-ce28e9ccaa7d h1:Ypy41GdYadUvX45qz4ux8BExKtPuBpSEeDNcyHhiOEg=

mapper/mapper.go

@@ -29,7 +29,7 @@ func ptrConverter(c context.Context, isPtr bool, sType reflect.Type, out reflect
 	}
 	k := out.Type().Kind()
 	if (k == reflect.Ptr || k == reflect.Interface) && out.IsNil() {
-		return nil, fmt.Errorf("Attempting to return nil for non-pointer type %v", sType)
+		return nil, fmt.Errorf("attempting to return nil for non-pointer type %v", sType)
 	}
 	return out.Interface(), nil
 }
@@ -52,7 +52,7 @@ func MakeBuilder(c context.Context, sType reflect.Type) (Builder, error) {
 
 	if sType.Kind() == reflect.Map {
 		if sType.Key().Kind() != reflect.String {
-			return nil, errors.New("Only maps with string keys are supported")
+			return nil, errors.New("only maps with string keys are supported")
 		}
 		return func(cols []string, vals []interface{}) (interface{}, error) {
 			out, err := buildMap(c, sType, cols, vals)
@@ -137,7 +137,7 @@ func buildMap(c context.Context, sType reflect.Type, cols []string, vals []inter
 		if rv.Elem().Elem().Type().ConvertibleTo(sType.Elem()) {
 			out.SetMapIndex(reflect.ValueOf(v), rv.Elem().Elem().Convert(sType.Elem()))
 		} else {
-			return out, fmt.Errorf("Unable to assign value %v of type %v to map value of type %v with key %s", rv.Elem().Elem(), rv.Elem().Elem().Type(), sType.Elem(), v)
+			return out, fmt.Errorf("unable to assign value %v of type %v to map value of type %v with key %s", rv.Elem().Elem(), rv.Elem().Elem().Type(), sType.Elem(), v)
 		}
 	}
 	return out, nil
@@ -185,7 +185,7 @@ func buildStructInner(c context.Context, sType reflect.Type, out reflect.Value,
 			field.Elem().Set(rv.Elem().Elem().Convert(curFieldType.Elem()))
 		} else {
 			logger.Log(c, logger.ERROR, fmt.Sprintln("can't find the field"))
-			return fmt.Errorf("Unable to assign pointer to value %v of type %v to struct field %s of type %v", rv.Elem().Elem(), rv.Elem().Elem().Type(), sf.name[depth], curFieldType)
+			return fmt.Errorf("unable to assign pointer to value %v of type %v to struct field %s of type %v", rv.Elem().Elem(), rv.Elem().Elem().Type(), sf.name[depth], curFieldType)
 		}
 	} else {
 		if reflect.PtrTo(curFieldType).Implements(scannerType) {
@@ -202,12 +202,12 @@ func buildStructInner(c context.Context, sType reflect.Type, out reflect.Value,
 			}
 		} else if rv.Elem().IsNil() {
 			logger.Log(c, logger.ERROR, fmt.Sprintln("Attempting to assign a nil to a non-pointer field"))
-			return fmt.Errorf("Unable to assign nil value to non-pointer struct field %s of type %v", sf.name[depth], curFieldType)
+			return fmt.Errorf("unable to assign nil value to non-pointer struct field %s of type %v", sf.name[depth], curFieldType)
 		} else if rv.Elem().Elem().Type().ConvertibleTo(curFieldType) {
 			field.Set(rv.Elem().Elem().Convert(curFieldType))
 		} else {
 			logger.Log(c, logger.ERROR, fmt.Sprintln("can't find the field"))
-			return fmt.Errorf("Unable to assign value %v of type %v to struct field %s of type %v", rv.Elem().Elem(), rv.Elem().Elem().Type(), sf.name[depth], curFieldType)
+			return fmt.Errorf("unable to assign value %v of type %v to struct field %s of type %v", rv.Elem().Elem(), rv.Elem().Elem().Type(), sf.name[depth], curFieldType)
 		}
 	}
 	return nil
@@ -223,7 +223,7 @@ func buildPrimitive(c context.Context, sType reflect.Type, cols []string, vals [
 	if rv.Elem().Elem().Type().ConvertibleTo(sType) {
 		out.Set(rv.Elem().Elem().Convert(sType))
 	} else {
-		return out, fmt.Errorf("Unable to assign value %v of type %v to return type of type %v", rv.Elem().Elem(), rv.Elem().Elem().Type(), sType)
+		return out, fmt.Errorf("unable to assign value %v of type %v to return type of type %v", rv.Elem().Elem(), rv.Elem().Elem().Type(), sType)
 	}
 	return out, nil
 }

mapper_test.go

@@ -198,8 +198,8 @@ func TestBuildSqlitePrimitiveNilFail(t *testing.T) {
 	if err == nil {
 		t.Error("Expected error didn't get one")
 	}
-	if err.Error() != "Attempting to return nil for non-pointer type string" {
-		t.Errorf("Expected error message '%s', got '%s'", "Attempting to return nil for non-pointer type string", err.Error())
+	if err.Error() != "attempting to return nil for non-pointer type string" {
+		t.Errorf("Expected error message '%s', got '%s'", "attempting to return nil for non-pointer type string", err.Error())
 	}
 
 	s, err = mapRows(c, rows, b)

proteus.go

@@ -67,8 +67,8 @@ var rw sync.RWMutex
 
 func SetLogLevel(ll logger.Level) {
 	rw.Lock()
-	defer rw.Unlock()
 	l = ll
+	rw.Unlock()
 }
 
 // ShouldBuild works like Build, with two differences:
@@ -78,25 +78,25 @@ func SetLogLevel(ll logger.Level) {
 // 2. All errors found during building will be reported back
 //
 // 3. The context passed in to ShouldBuild can be used to specify the logging level used during ShouldBuild and
-// when the generated functions are invoked. If a logging level was specified using the SetLogLevel function,
-// the log level specifed will override any log level in the context.
+// when the generated functions are invoked. This overrides any logging level specified using the SetLogLevel
+// function.
 func ShouldBuild(c context.Context, dao interface{}, paramAdapter ParamAdapter, mappers ...QueryMapper) error {
-	//if log level is set, then override the log level specified here
-	rw.RLock()
-	defer rw.RUnlock()
-	if l != logger.OFF {
+	//if log level is set and not in the context, use it
+	if _, ok := logger.LevelFromContext(c); !ok && l != logger.OFF {
+		rw.RLock()
 		c = logger.WithLevel(c, l)
+		rw.RUnlock()
 	}
 
 	daoPointerType := reflect.TypeOf(dao)
 	//must be a pointer to struct
 	if daoPointerType.Kind() != reflect.Ptr {
-		return errors.New("Not a pointer")
+		return errors.New("not a pointer")
 	}
 	daoType := daoPointerType.Elem()
 	//if not a struct, error out
 	if daoType.Kind() != reflect.Struct {
-		return errors.New("Not a pointer to struct")
+		return errors.New("not a pointer to struct")
 	}
 	var out error
 	funcs := make([]reflect.Value, daoType.NumField())
@@ -172,17 +172,17 @@ func ShouldBuild(c context.Context, dao interface{}, paramAdapter ParamAdapter,
 // Build is the main entry point into Proteus
 func Build(dao interface{}, paramAdapter ParamAdapter, mappers ...QueryMapper) error {
 	rw.RLock()
-	defer rw.RUnlock()
 	c := logger.WithLevel(context.Background(), l)
+	rw.RUnlock()
 	daoPointerType := reflect.TypeOf(dao)
 	//must be a pointer to struct
 	if daoPointerType.Kind() != reflect.Ptr {
-		return errors.New("Not a pointer")
+		return errors.New("not a pointer")
 	}
 	daoType := daoPointerType.Elem()
 	//if not a struct, error out
 	if daoType.Kind() != reflect.Struct {
-		return errors.New("Not a pointer to struct")
+		return errors.New("not a pointer to struct")
 	}
 	daoPointerValue := reflect.ValueOf(dao)
 	daoValue := reflect.Indirect(daoPointerValue)
@@ -194,7 +194,10 @@ func Build(dao interface{}, paramAdapter ParamAdapter, mappers ...QueryMapper) e
 		//recurse
 		if curField.Type.Kind() == reflect.Struct && curField.Anonymous {
 			pv := reflect.New(curField.Type)
-			Build(pv.Interface(), paramAdapter, mappers...)
+			err := Build(pv.Interface(), paramAdapter, mappers...)
+			if err != nil {
+				return err
+			}
 			daoValue.Field(i).Set(pv.Elem())
 			continue
 		}