samber / lo Goto Github PK

I just learned about this project (been using RxGo), and the API and helpers look really useful.

I mostly work on streaming-based systems/APIs in Go so was wondering if you're also considering adding support for such use cases?

E.g. filter and map based on a stream (Go channel?) of Order structs coming from a messaging system (Kafka, Rabbit, Redis) or HTTP (gRPC) poll/push API.

Your naive shuffle algorithm suffers the common over shuffling problem

https://possiblywrong.wordpress.com/2014/12/01/card-shuffling-algorithms-good-and-bad/

Go uses the https://en.wikipedia.org/wiki/Fisher–Yates_shuffle standard algorithm.

But you can just use Go's implementation from rand.

func Shuffle[T any](collection []T) []T {
	rand.Shuffle(len(collection), func(i, j int) {
		collection[i], collection[j] = collection[j], collection[i]
	})
	return collection
}

Create function Count[T any](collection []T, predicate func (T) bool) int would return the number of elements for which the predicate returns true.

Sometimes if the collection is too big, it will make too many goroutines in one time when using parallel methods. That will significantly increase resource usage, especially when the iteratee function is computationally-Intensive, io-Intensive, or both.

An example that requests every URL in a huge list, will start so many requests at the same time:

hugeUrlList := []string{ /*...*/ }
parallel.ForEach(hugeUrlList, request)

I think there should have an option to set the max concurrency count, which means up to configured number iteratee calling at the same time:

hugeUrlList := []string{ /*...*/ }
parallel.ForEach(hugeUrlList, request, parallel.Option().Concurrency(100))

I will enjoy implementing it.

Function retrieves any slices and a callback which returns 2 values: map key and map value.

Possible implementation:

func ToMap[S, V any, K comparable, M ~map[K]V](s []S, fn func(S) (K, V)) M {
	m := make(map[K]V, len(s))
	for _, elem := range s {
		key, value := fn(elem)
		m[key] = value
	}
	return m
}

Create function func MaxBy[T any](collection []T, comparison func (T,T) bool) T that returns the max using the given comparison function.
Create function func MinBy[T any](collection []T, comparison func (T,T) bool) T that returns the min using the given comparison function.

It might be convenient to have a slice function similar to Find(), but that you can pass a default value to return if no match was found. Signature could look like so:

FindDefault[T any](collection []T, fallback T, predicate func(T) bool) T

when calling min with an empty slice it should return the maximum value
when calling max with an empty slice it should return the minimum value

Lodash describes this operation as returning unique entries
https://lodash.com/docs/#intersection

Also described in SQL:
https://www.postgresql.org/docs/9.4/queries-union.html

INTERSECT returns all rows that are both in the result of query1 and in the result of query2. Duplicate rows are eliminated unless INTERSECT ALL is used.

EXCEPT returns all rows that are in the result of query1 but not in the result of query2. (This is sometimes called the difference between two queries.) Again, duplicates are eliminated unless EXCEPT ALL is used.

https://docs.microsoft.com/en-us/sql/t-sql/language-elements/set-operators-except-and-intersect-transact-sql?view=sql-server-ver15

Say you have a collection and want to find the element that minimizes or maximizes a function.

For example, here I have a slice of person structs and am looking for the one for which the computeAge method returns the lowest value:

type person struct {
	name     string
	birthday time.Time
}

func (p *person) computeAge() int {
	age := time.Now().Year() - p.birthday.Year() - 1
	if p.birthday.YearDay() < time.Now().YearDay() {
		age += 1
	}
	return age
}

func main() {
	people := []person{
		{"Jordan", time.Date(1994, time.October, 2, 0, 0, 0, 0, time.UTC)},
		{"Emily", time.Date(1868, time.March, 26, 0, 0, 0, 0, time.UTC)},
		{"Micheal", time.Date(1999, time.June, 5, 0, 0, 0, 0, time.UTC)},
		{"Nikola", time.Date(2001, time.November, 16, 0, 0, 0, 0, time.UTC)},
		{"Rachel", time.Date(1993, time.April, 1, 0, 0, 0, 0, time.UTC)},
	}

	youngest := lo.MinBy(people, func(a, b person) bool {
		return a.computeAge() < b.computeAge()
	})
	fmt.Println(youngest.name)
}

Using MinBy like this results in 8 calls to computeAge where 5 should have sufficed. Using a custom keying function with MinBy results in many more calls than necessary.

I propose adding a MinByKey and MaxByKey functions that apply the custom keying function to all elements and then return the one that minimizes/maximizes it.

The signatures would be:

func MinByKey[T any, K constraints.Ordered](collection []T, key func(T) K) T
func MaxByKey[T any, K constraints.Ordered](collection []T, key func(T) K) T

Would this contribution be appreciated @samber ?

ContainsBy example has Contains title.

It seems that lo can't do any chain opertaions like

type Poo struct {
	Id   int
	Name string
}

poos := []Poo{
		{1, "A"},
		{2, "B"},
		{3, "C"},
	}


//  it dosen't work 
lo.Map[Poo, int](poos, func(x Poo, _ int) int {
		return x.Id
	}).Filter[int](lo.Map[Poo, int](poos, func(x int, _ int) bool {
		return x%2 == 0
	})

// it works 
even := lo.Filter[int](lo.Map[Poo, int](poos, func(x Poo, _ int) int {
		return x.Id
	}), func(x int, _ int) bool {
		return x%2 == 0
	})

It will be supported in the future?
May be we can introduce a middle data stream like the stream operation in Java

In:  Intersect[int]([]int{0, 6, 0}, []int{0, 1, 2, 3, 5, 5, 6, 6})
Out: []int{0, 6, 6}

In:  Difference[int]([]int{0, 1, 2, 3, 3, 4, 4}, []int{0, 1, 2, 2, 3, 5, 5})
Out: []int{4, 4}, []int{5, 5}

In:  Union[int]([]int{0, 1, 2, 3, 3, 5, 6, 6}, []int{0, 1, 2, 4, 4, 5, 5, 6})
Out: []int{0, 1, 2, 3, 3, 5, 6, 6, 4, 4}

Hello, these functions are dealing with duplicate values differently. Is there a rule that the collections disallow duplicates?

Current implementation spawn as many go-routines as passed collection size (for example as passed to Map) method in parallel package. While this is perfectly okay to do for up to a few thousand go-routines, it could become problematic for larger values. For example, for a mapping operation that makes any networking call, this is almost equivalent to DOSsing the end service.

Suggestion is to allow configuring concurrency for such parallel operations. Default would be equal to runtime.NumCPU(), whereas, if anything <= 0 is specified, it would automatically switch to current default behavior. Anything greater than 0 would override previous behaviors specified.

Sample implementation with above in mind for Map method can be found here (this diff is against current implementation): https://www.diffchecker.com/4pLAUeMh

Let me know if this looks good. I can work on the PR.

Best wishes...

var someTime = Must(time.Parse("2006-01-02", "2022-01-15"))

Must allows converting unexpected errors into panics, very useful when initializing global variables. This is a pretty common pattern and lot of libraries ship their own implementation of it eg:

Must in "text/template"
Must in "html/template"
MustParse in "github.com/google/uuid"
MustCompile in "regexp"

Lots more examples as well.

Additionally to error returns, you also often stumble upon boolean returns.

Fictional example:

func Parse(raw string) (out ConcreteType, ok bool)

It is possible to use type switches within generic function, so we could switch for error and bool alike.
Allowing the Must() function to become slightly more generic than it is right now without breaking anything.

It would be useful to have:
func Duplicate[T comparable](collection []T) []T
func DuplicateBy[T any, U comparable](collection []T, iteratee func(T) U) []T

That would only return the duplicated values of the collection
.

It would be great to be able to add context to the panic message in Must.
That would be especially useful in the case where Must validates a boolean as at the moment it panics with the message "not ok".

I was thinking we could add the parameter msgAndArgs ...interface{} at the end of each Must function like it is done in the assert functions of the testify package.

We could then call it like that for example:

lo.Must0(strings.Contains(myString, requiredSubString)), "'%s' must always contain '%s'", myString, requiredSubString)

Note:
One issue with the solution described above is that the compiler will not detect when the number of values returned by the function used in Must doesn't match the expected number of parameters of Must.

One solution for that could be to create new function like MustF & MustFX which take a string instead of msgAndArgs ...interface{}

For the same example we would then have:

lo.MustF0(strings.Contains(myString, requiredSubString)), fmt.Sprintf("'%s' must always contain '%s'", myString, requiredSubString))

Is this project going to be actively maintained?

I see that people would like to contribute, but issues and PR's review and acceptance depends only on @samber and his free time.
May be such bus-factor can be reduced somehow?

See https://play.golang.com/p/YgPwRAtSQdb

package main

import (
	"fmt"
	
	"github.com/samber/lo"
)

func main() {
	ints := []int{0,1,2}
	fmt.Println(lo.Nth(ints, -1))  // prints 2, as expected
	
	ints = []int{0}
	fmt.Println(lo.Nth(ints, -1))  // returns error "nth: -1 out of slice bounds" - but shouldn't
}

Additionally, I think this function should panic instead of raising an error when out of bounds (but I'm open to be convinced otherwise):

• the original lodash implementation doesn't return an error (returns undefined when out of bounds)
• normal Go slice accesses panic when out of bounds

I'll submit a PR with a fix - and my proposed change, but wanted to file this issue first.

Similar to https://github.com/luraim/fun#filtermap, performing filtering and mapping in a single call.

Since with go 1.18 release new slices and maps packages were introduced it makes sense to use them under the hood instead of self-written functions.

What do you think?

It would be useful to have functions like that:

func UnpackT2[A any, B any](tuple Tuple2[A, B]) (A, B){
    return tuple.A, tuple.B
}

That would allow to extract variables from tuples which would make the code more readable in some situations.

I can provide a PR for this.

All[T any](collection []T, predicate func (T) bool) bool would return true if all the elements pass the predicate and short-circuit to false otherwise.

Any[T any](collection []T, predicate func (T) bool) bool would return false if none the elements pass the predicate and short-circuit to true otherwise.

These are common functions in Rust, Haskell, many other languages.

Hi,
Moving from go-funk to lo, just noticed go-funk is still included in my dependency graph, probably due to it being a dependency for the benchmark suite in lo.
Not sure, but is there any way to remove it from the 'release' lib?

Needed to read the description to get what it does. Naming it Merge would make it obvious. What do you think?

I'm new to Go lang community, so this is most a discussion about if it'd be a good practice than a suggestion by itself. Coming from the Javascript world, it is common to see situations where this expression is used:

function (arg1, arg2) {
    return arg1 ?? arg2 ?? "[default value]"
}

In SQL databases it is also common to find the Coalesce function that solves a similar problem:

SELECT COALESCE(column1, column2, '[default value]')
FROM xyz

But I didn't find in Go a simple way to handle nil values and use a default value instead, needing to use the Ternary() or Find() functions as a solution, but still they don't seem to be the most idiomatic way to solve the problem. Following examples above, it might be interesting to have functions like:

Coalesce(): Returns the first non-nil argument passed to the function
NullIf(): Returns nil if both arguments are equal, otherwise returns the first

It would be nice if there something like https://www.geeksforgeeks.org/lodash-_-pipeline-method.

Like above, and is this library not compatible with others?

@samber I may suggest a PR if you agreed with this proposal.

We could add a function that execute a function in a goroutine and returns the result in a channel:

func Async[T any](f func() T) chan T {
	channel := make(chan T)
	go func() {
		channel <- f()
	}()
	return channel
}

Let's say there's an iteratee:

func iteratee(a int) int {
	return a + 1
}

Whenever I use functions in lo, it requires me to write another iteratee to wrap it:

slice = lo.Map(slice, func(i int, _ int) int {
	return iteratee(i)
})

Well, it's acceptable, but not elegant. If lo could provide a function like this:

func WrapIndex[T any, R any](f func(T) R) func(T, int) R {
	return func(t T, _ int) R {
		return f(t)
	}
}

Then the function could be written in this way:

slice = lo.Map(slice, WrapIndex(iteratee))

It's seldom we use an indexed version iteratee, most of the time we just throw away the index.

However, providing an unindexed version for all functions would be tedious. Instead, providing a wrapper function to throw away the index should be better.

BTW, I don't know whether I used the words 'indexed' and 'unindexed' correctly. Hope you could understand. XD

Can we get some more common string operations, such as the classic chop (force one rune truncation) and chomp (remove trailing runes only on CR, LF, or CRLF match)?

Such function could be named like ValuesF which means Values Filtered or may be you suggest another naming.

The implementation is just an extension of the std lib slices.Values function:

ValuesF[K comparable, V any](m ~map[K]V, keys []K) []V {
	r := make([]V, 0, len(m))
	for k, v := range m {
		if slices.Contains(keys, k) {
			r = append(r, v)
		}
	}
	return r
}

Let's talk about a new suite of helpers for manipulating channels.

// ToChannel returns a read-only channels of collection elements.
func ToChannel[T any](collection []T) <-chan T

// Generator implements the generator design pattern.
func Generator[T any](bufferSize int, generator func(int64) T) <-chan T

// Batch creates a slice of n elements from a channel. Returns the slice and the slice length.
func Batch[T any](ch <-chan T, size int) (collection []T, length int)

// BatchWithTimeout creates a slice of n elements from a channel, with timeout. Returns the slice and the slice length.
func BatchWithTimeout[T any](ch <-chan T, size int, timeout time.Duration) (collection []T, length int)

Some thoughts about BatchXXX functions:

• return channel status: ok bool?
• return batch time: duration time.Duration?
• return <-chan []T instead of []T ?
• accept a buffer as a parameter instead of repeated allocation?
• allocate channel with buffer size > 1 ?

Quite frequently, I want to do something like lo.Map(slice, Transformer) but Transformer returns (R, error).

I can write a function literal and handle the error, but it's still not ergonomic as there's no proper way to return the error.

So, I propose (and I can submit a PR for this) a variant of Map (and maybe others that make sense) that looks like this:

func Map[T any, R any](collection []T, iteratee func(T, int) (R, error)) ([]R, error)

Where, if the iteratee returns an error, the entire iteration stops, returns an empty list and an error (or maybe returns what it has so far)

Invert is nondeterministic when a map contains duplicate values because map iteration is randomized by the go compiler.

I don't think this function really should exist. But I'm curious if anyone is using/relying on this, and what the use case is.

This is a request for a new function: Except. This would work similar to Difference except that it documents (assumes) that list1 is a superset of list2. This is the logical inverse of the Intersect().

I understand that Difference actually has this behavior:

https://go.dev/play/p/WxfAXrK70wX

From a user experience perspective, it does a bit more than I need (I have to throw away the second array). And then there's this:

I'm happy to create a PR. Please let me know if you would like this. I believe that Except would be slightly faster, and with a slightly smaller memory footprint as well.

Thank you for this awesome library!!

Some languages like Typescript or Swift offer a very cool syntactic sugar a?.b?.c?.d?.e.

In Go, we need to write a condition similar to: a != nil && a.b != nil && a.b.c != nil.

I create this issue for discussing a new helper.

In PR #106, I suggest an implementation called Safe:

type a struct {
	foo *string
}
type b struct {
	a *a
}
type c struct {
	b *b
}

v := &c{
	b: &b{
		a: nil,
	},
}

foo, ok := lo.Safe(func() string { return *v.b.a.foo })
// "", false

Calling *v.b.a.foo will panic, but this nil pointer error will be caught by lo.Safe. Other exception won't be caught.

This implementation is much more "typesafe" than something like lo.Safe(a, ".b.c.d")

WDYT?

like this

type Foo struct {
    ID string
}
value := []*Foo{{ID: "a"}, {ID: "b"}, {ID: "c"}}
pipe := lo.NewPipeline(value)
result := pipe.Map(func(v *Foo) string { return v.ID }).Filter(func(v *Foo) bool { return v.ID == "a" }).Uniq().Result()

It would be useful to have functions ToTupleX to simplify the creation of Tuples.

It would be especially useful to convert the return of a function with multiple values into a single Tuple.

func MinBy[T any](collection []T, comparison func(T, T) bool) T {
	var min T

	if len(collection) == 0 {
		return min
	}

	min = collection[0]

	for i := 1; i < len(collection); i++ {
		item := collection[i]

		if comparison(item, min) {
			min = item
		}
	}

	return min
}

func MaxBy[T any](collection []T, comparison func(T, T) bool) T {
	var max T

	if len(collection) == 0 {
		return max
	}

	max = collection[0]

	for i := 1; i < len(collection); i++ {
		item := collection[i]

		if comparison(item, max) {
			max = item
		}
	}

	return max
}

comparison in MinBy is less，while in MaxBy is greater. In most of the generics library, just need one comparison function, less, such as C++ STL. This minimizes user misuse.

The Find function exists. The IndexOf function exists. What would be super helpful is a FindIndex function; similar in mechanics to Find, except it returns the slice index of the first slice member which satisfies the Find function (or -1 if none are found).

Awesome project, by the way!

I use this function personally. With this, you can save dozens of lines with if err != nil. Execute all simple logics (doSimpleThing) first, and check the error later. The error will be the first occured error or nil.

func Delay[T any](target *error) func(v T, err error) T {
	a := func(v T, err error) T {
		if *target == nil && err != nil {
			*target = err
		}
		return v
	}
	return a
}

Example:

func doSimpleThing1() (int, error){...}
func doSimpleThing2() (string, error){...}

var err error

dInt := Delay[int](&err)
dString := Delay[string](&err)

_ = complexStruct{
    id: dInt(doSimpleThing1()),
    name: dString(doSimpleThing2()),
    email: dString(doSimpleThing2()),
    ...
}

// check error later
if err != nil {
    panic(err)
}

I think this function is appropriate for this package if there's a good name for it. Do you have a good idea?

Currently, description of Empty() function is missing in README.

https://github.com/samber/lo/blob/master/pointers.go#L16

Example:

even, odd := lo.FilterXXX[int]([]int{1, 2, 3, 4}, func(x int, _ int) bool {
    return x%2 == 0
})

I would name it lo.Filter2 but it looks weird. Does anyone have a better idea?

A lo.RejectXXX helper would be excellent too.