Intelligent Contracts
Image Processing

Image Processing

Image Processing in Intelligent Contracts lets contracts send screenshots, photos, or other visual data to LLMs alongside a prompt for analysis.

Sending Images to LLMs

Use the images parameter in gl.nondet.exec_prompt():

from genlayer import *
 
class ReceiptVerifier(gl.Contract):
    verified: bool
 
    def __init__(self):
        self.verified = False
 
    @gl.public.write
    def verify_receipt(self, image_data: bytes, expected_amount: str) -> None:
        def leader_fn():
            return gl.nondet.exec_prompt(
                f"Does this receipt show a payment of {expected_amount}? "
                "Respond as JSON: {{\"matches\": true/false, \"actual_amount\": \"...\"}}",
                images=[image_data],  # accepts raw bytes directly
                response_format="json",
            )
 
        def validator_fn(leaders_res) -> bool:
            if not isinstance(leaders_res, gl.vm.Return):
                return False
            my_result = leader_fn()
            return my_result["matches"] == leaders_res.calldata["matches"]
 
        result = gl.vm.run_nondet_unsafe(leader_fn, validator_fn)
        self.verified = result["matches"]

The images parameter accepts a sequence of raw bytes (e.g., PNG/JPEG data) or gl.nondet.Image objects.

Sending Multiple Images

Pass both images in the same images list to compare them in one prompt. The limit is two images per call. Refer to them in list order so the model can distinguish the first image from the second.

This helper can be called from a contract method with two complete PNG/JPEG byte sequences, such as a product photo and a reference photo:

def compare_images(product_image: bytes, reference_image: bytes) -> bool:
    def leader_fn():
        result = gl.nondet.exec_prompt(
            "Does the product in the first image match the product in the second image? "
            'Respond as JSON: {"matches": true} or {"matches": false}.',
            images=[product_image, reference_image],
            response_format="json",
        )
        if not isinstance(result, dict) or not isinstance(result.get("matches"), bool):
            raise gl.vm.UserError("Expected a boolean matches field")
        return result["matches"]
 
    def validator_fn(leaders_res) -> bool:
        if not isinstance(leaders_res, gl.vm.Return):
            return False
        return isinstance(leaders_res.calldata, bool) and leader_fn() == leaders_res.calldata
 
    return gl.vm.run_nondet_unsafe(leader_fn, validator_fn)

Each validator independently compares the images and checks the leader's boolean decision. Ambiguous photos can produce disagreement; choose a precise comparison criterion for your application. See Calling LLMs for response validation patterns.

Capturing Screenshots from the Web

Combine web access with image processing to screenshot a webpage and analyze it:

def check_website_status():
    url = "https://example.com/status-page"
    screenshot = gl.nondet.web.render(url, mode='screenshot')
    
    return gl.nondet.exec_prompt(
        "Is this status page showing all systems operational? "
        "Respond as JSON: {{\"all_operational\": true/false}}",
        images=[screenshot],
        response_format="json",
    )
 
result = gl.eq_principle.strict_eq(check_website_status)

Use Cases

  • Visual evidence verification — insurance claims with photo proof, damage assessment
  • Document analysis — receipts, invoices, certificates
  • Web monitoring — screenshot a page and verify its content matches expectations
  • Brand & content compliance — check if visual content meets guidelines
  • UI verification — screenshot an app and verify it renders correctly

Image processing requires vision-capable LLM models. On the GenLayer network, validators handle model selection — your contract just sends images. In Studio running locally, ensure your configured validators use a model that supports image inputs (e.g., GPT-5, Claude Sonnet).