Support for negative prompts in Swift CLI

[mfellner]

As far as I could see, the Swift CLI implementation currently doesn't support negative prompts.

In the Python implementation, the negative prompt argument is tokenised, encoded into uncond_embeddings, and prepended to the text embeddings of the regular prompt:

ml-stable-diffusion/python_coreml_stable_diffusion/pipeline.py

Lines 145 to 165 in 583cc04

	else:
	uncond_tokens = negative_prompt
	max_length = text_input_ids.shape[-1]
	uncond_input = self.tokenizer(
	uncond_tokens,
	padding="max_length",
	max_length=max_length,
	truncation=True,
	return_tensors="np",
	)
	uncond_embeddings = self.text_encoder(
	input_ids=uncond_input.input_ids.astype(
	np.float32))["last_hidden_state"]
	# For classifier free guidance, we need to do two forward passes.
	# Here we concatenate the unconditional and text embeddings into a single batch
	# to avoid doing two forward passes
	text_embeddings = np.concatenate(
	[uncond_embeddings, text_embeddings])

The Swift implementation on the other hand does not handle a negative prompt argument:

ml-stable-diffusion/swift/StableDiffusion/pipeline/StableDiffusionPipeline.swift

Lines 76 to 84 in 583cc04

	// Encode the input prompt as well as a blank unconditioned input
	let promptEmbedding = try textEncoder.encode(prompt)
	let blankEmbedding = try textEncoder.encode("")
	// Convert to Unet hidden state representation
	let concatEmbedding = MLShapedArray<Float32>(
	concatenating: [blankEmbedding, promptEmbedding],
	alongAxis: 0
	)

I don't really know how Stable Diffusion (v2) deals with negative prompts but how would one add support for them to the Swift implementation? The Core ML model seems to support it after all.

[JustinMeans]

I just opened a PR with the Swift implementation. See: #61