Effect.gen

So far we’ve seen how to construct an Effect, transform an Effect, and run an Effect. We’ve seen that Effect.flatMap allows us to combine two Effects by running them in sequence, but the syntax was a little clunky. It was a bit like working with Promise.then, leading to nested function execution and something resembling callback hell. Effect.gen makes it simpler to work with multiple Effects, a bit like how async/await syntax makes it easier to work with Promises.

Let’s see an example.

import { Data, Effect } from "effect";

const random = Effect.sync(() => Math.random());
const roll = random.pipe(Effect.map((v) => Math.ceil(v * 6)));

class YouLose extends Data.TaggedError("YouLose")<{}> {}

const program = Effect.gen(function* () {
	const first = yield* roll;
	const second = yield* roll;
	const total = first + second;
	if (total < 5) {
		yield* new YouLose();
	}
	return total;
});

program.pipe(Effect.map(console.log), Effect.runSync);

Even if the syntax is unfamiliar, I hope the intent of the code is clear. If you roll 5 or above you win, otherwise you lose. Before we dive into the guts, let’s compare this against what this would look like without Effect.gen.

const equivalent = roll.pipe(
	Effect.flatMap((first) => roll.pipe(Effect.map((second) => first + second))),
	Effect.flatMap((total) => {
		if (total < 5) {
			return new YouLose();
		}
		return Effect.succeed(total);
	}),
);

This code works, and it is OK, but already there are some readability issues. There are multiple levels of nesting. Although the code runs from top to bottom, that is not entirely clear at first glance. Refactoring this code to add 3 dice rolls would require some thought. If we wanted to write even more dynamic code, such as “flip a coin to decide how many dice to roll, then return the total of the dice rolls”, it would quickly become hard to maintain.

I won’t give a detailed explanation of how generators work in JavaScript, but here is a whistle-stop tour. A generator function is written using function* syntax. There is no arrow function equivalent for generator functions. A generator function may yield one or more values. It may also return a value. When the function is called, it returns a generator. The generator has a few methods, most notably .next(). Let’s see how this works.

// makeGenerator is a Generator Function
const makeGenerator = function* () {
	yield "truly";
	yield "madly";
	yield "deeply";
};

// first is a Generator
const first = makeGenerator();
console.log(first.next()); // { value: 'truly', done: false }
console.log(first.next()); // { value: 'madly', done: false }
console.log(first.next()); // { value: 'deeply', done: false }
console.log(first.next()); // { value: undefined, done: true }
console.log(first.next()); // { value: undefined, done: true }

When you call a generator function you initialise the generator at the start of the function body. When you call .next() the execution will proceed to the next yield, at which point it pauses (“yields”) execution back to the caller. The return value from .next() will be the value of the yield expression, wrapped as an object. Each .next() call will proceed to the next yield, until the function body has been exhausted.

Note that each time we call the generator function it creates a new generator. Each generator progresses through the function body independently.

const second = makeGenerator();
console.log(second.next()); // { value: 'truly', done: false }
console.log(second.next()); // { value: 'madly', done: false }
console.log(first.next()); // { value: undefined, done: true }

The values from a finite generator can be collected into an Array. You can loop over a generator using for ... of syntax.

console.log(Array.from(makeGenerator())); // [ 'truly', 'madly', 'deeply' ]
for (let value of makeGenerator()) {
    console.log(value);
}

You need to be careful with this. It is possible - and often very useful - to define infinite generators.

// not all generators are finite
const makeInfiniteCounter = function* () {
	let count = 0;
	while (true) {
		yield count;
		count += 1;
	}
};

const counter = makeInfiniteCounter();
console.log(counter.next()); // { value: 0, done: false }
console.log(counter.next()); // { value: 1, done: false }
console.log(counter.next()); // { value: 2, done: false }
console.log(counter.next()); // { value: 3, done: false }
console.log(counter.next()); // { value: 4, done: false }

It is possible to make new generators from other generators. In particular, the take function lets us take a fixed number of elements from a generator. Here’s one possible implementation.

const take = function* <T>(times: number, gen: Generator<T>) {
	const counter = makeInfiniteCounter();
	while (counter.next().value < times) {
		const retrieved = gen.next();
        console.debug("[debug]", retrieved);
		if (retrieved.done) {
			return;
		}
		yield retrieved.value;
	}
};

const firstThree = take(3, makeInfiniteCounter());
const shorterThanExpected = take(5, makeGenerator());

console.log(Array.from(firstThree)); // [ 0, 1, 2 ]
console.log(Array.from(shorterThanExpected)); // [ 'truly', 'madly', 'deeply' ]

Generators - including take - are lazy. Values are only produced when needed. In this case, that is the array conversion. This laziness means that generators are composable.

For example, we can use the above building blocks to build other common functionality, like repeating a known valoue.

const repeat = function* <T>(times: number, value: T) {
	const counter = take(times, makeInfiniteCounter());
	for (let _ of counter) {
		yield value;
	}
};

const matthew = repeat(3, "alright");
console.log(Array.from(matthew)); // [ 'alright', 'alright', 'alright' ]

You can call a generator inside another generator.

const nestedPlain = function* () {
	const counter = makeInfiniteCounter();
	for (let count of counter) {
		for (let repeated of repeat(count, count)) {
			yield repeated;
		}
	}
};

const plainItems = take(10, nestedPlain());
console.log(Array.from(plainItems).join(" ")); // '1 2 2 3 3 3 4 4 4 4'

This delegation logic is so common that it has dedicated syntax through the yield* operator. This really helps with generator composition.

const nestedStar = function* () {
	const counter = makeInfiniteCounter();
	for (let count of counter) {
		yield* repeat(count, count);
	}
};

const starItems = take(10, nestedStar());
console.log(Array.from(starItems).join(" ")); // '1 2 2 3 3 3 4 4 4 4'

We have seen that generator functions are lazy, composable, and can produce values of a known type. This makes them an ideal fit for Effect! Effect.gen wraps a generator function that produces Effects and upgrades it to be an Effect.

Here are a few more examples using Effect.gen.

get (from the Effect.tryPromise lesson). Observe how the “piped” version resembles Promise.then and the “gen” version resembles async/await. The generator helps us avoid “callback hell”, by allowing us to unwrap a variable. Inside the generator we are only considering the happy-path case. The generator will short circuit on the first error.

const getThen = (url: string) =>
    fetch(url).then(response => {
        if (!response.ok) {
            throw new ResponseFailure(response.statusText);
        }
        return response.json();
    });


const getAwait = (url: string) => async () => {
    const response = await fetch(url);
    if (!response.ok) {
        throw new ResponseFailure(response.statusText);
    }
    const json = await response.json();
    return json;
};

const getPipe = (url: string) =>
	Effect.tryPromise(() => fetch(url)).pipe(
		Effect.flatMap((response) => {
            if (!response.ok) {
                return Effect.fail(new ResponseFailure(response.statusText));
            }
			return Effect.tryPromise((): Promise<unknown> => response.json());
        }),
	);

const getGen = (url: string) => 
	Effect.gen(function* (_) {
		const response = yield* Effect.tryPromise(() => fetch(url));
        if (!response.ok) {
            throw new ResponseFailure(response.statusText);
        }
		const json = yield* Effect.tryPromise((): Promise<unknown> => response.json());
		return json;
	});

validateHostname (from the Effect.flatMap lesson). Note that in this case, where the validation logic is a series of transformations on one variable at a time, the generator version is not necessarily easier to read. We can mix and match pipe vs generator style. It mostly boils down to personal preference.

const validateHostnamePipe = (value: unknown) =>
	Effect.succeed(value).pipe(
		Effect.flatMap(isString),
		Effect.flatMap(minLength(5)),
		Effect.map((s) => s.toLowerCase()),
		Effect.flatMap(isUrl),
		Effect.map((url) => url.hostname),
	);

const validateHostnameGen = (value: unknown) =>
	Effect.gen(function* () {
		const string = yield* isString(value);
		yield* minLength(5)(string);
		const lowercase = string.toLowerCase();
		const url = yield* isUrl(lowercase);
		return url.hostname;
	});

Finally, let’s try to understand how Effect.gen works behind the scenes. Note that we can start using Effect.gen without understanding the gory details. Don’t worry if this final section doesn’t sink in.

We define a Program wrapper, to represent a structure similar to an Effect. We can add a few common functions, like succeed, flatMap and runSync.

class Program<A> {
	constructor(readonly wrapped: () => Generator<never, A>) {}

    // this implements the "Iterable" protocol for Program, which is 
    // what `yield* program` does behind the scenes
	[Symbol.iterator]() {
		return this.wrapped();
	}

	pipe = <A>(a: (t: this) => A) => {
		return a(this);
	};
}

const gen = <A>(fn: () => Generator<never, A>) => new Program(fn);

const succeed = <A>(value: A): Program<A> =>
	gen(function* () {
		return value;
	});

const sync = <A>(fn: () => A): Program<A> =>
	gen(function* () {
		return fn();
	});

const fail = (e: unknown): Program<never> =>
	gen(function* () {
		throw e;
	});

const runSync = <A>(program: Program<A>): A => {
	return program.wrapped().next().value;
};

const map =
	<A, B>(fn: (a: A) => B) =>
	(program: Program<A>): Program<B> =>
		gen(function* () {
			const result = yield* program;
			return fn(result);
		});

const flatMap =
	<A, B>(fn: (a: A) => Program<B>) =>
	(program: Program<A>): Program<B> =>
		gen(function* () {
			const first = yield* program;
			const second = yield* fn(first);
			return second;
		});

Now we can use our Program library similar to Effect.

const random: Program<number> = sync(() => Math.random());

const roll = random.pipe(map((v) => Math.ceil(v * 6)));
const youLose: Program<never> = fail(new Error("you lose"));

const programGen = gen(function* () {
	const first = yield* roll;
	const second = yield* roll;
	const total = first + second;
	if (total < 5) {
		yield* youLose;
	}
	return total;
});

const programPipe = roll
	.pipe(flatMap((first) => roll.pipe(map((second) => first + second))))
	.pipe(
		flatMap((total) => {
			if (total < 5) {
				return youLose;
			}
			return succeed(total);
		}),
	);

console.log(runSync(programGen));
console.log(runSync(programPipe));

This implementation is for illustrative purposes only. The real implementation is much more complex. In particular, we have directly coupled the definition of an operation with its interpretation. This means that runSync is essentially the only way we could possibly run the programs. Instead, as we saw in early lessons, the Effect library builds up Effects as tree-like plain objects. Each instruction leaf is then interpreted by a runtime, which behaves a bit like our generator.

Let’s recap:

• Effect.gen helps us write readable effectful code
• Generator functions are lazy and composable, which makes them a good fit for writing Effects
• Generator function syntax, using function* and yield, is similar to async/await syntax
• When writing effectful code we can use pipes, Effect.gen, or a mix of styles