Sign inSign up

affinefoundation/lgc

By affinefoundation

Updated 7 months ago

The environment evaluates language models on logical reasoning tasks

Image
Machine learning & AI
0

2.4K

affinefoundation/lgc repository overview

Logic Environment (LGC)

Task Description

The Logic (LGC) environment evaluates language models on logical reasoning tasks from the Intellect-3-RL dataset. Tasks cover various types of logic puzzles and reasoning challenges that test formal reasoning capabilities.

How to Use with Affinetes

Option 1: Pull Pre-built Docker Image
import affinetes as af
import asyncio
import os

async def main():
    # Load environment from Docker Hub
    env = af.load_env(
        image="affinefoundation/lgc:pi",
        env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
    )
    
    # Evaluate on a logic task
    result = await env.evaluate(
        model="deepseek-ai/DeepSeek-V3",
        base_url="https://llm.chutes.ai/v1",
        task_id=42,  # Optional: index into dataset
    )
    
    print(f"Score: {result['score']}")
    print(f"Success: {result['success']}")
    
    await env.cleanup()

asyncio.run(main())
Option 2: Build from Source
import affinetes as af
import asyncio
import os

async def main():
    # Build image from local environment directory
    image_tag = af.build_image_from_env(
        env_path="environments/primeintellect/lgc",
        image_tag="lgc:pi"
    )
    
    # Load environment
    env = af.load_env(
        image=image_tag,
        env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
    )
    
    # Evaluate on a logic task
    result = await env.evaluate(
        model="deepseek-ai/DeepSeek-V3",
        base_url="https://llm.chutes.ai/v1",
        task_id=42
    )
    
    print(f"Score: {result['score']}")
    
    await env.cleanup()

asyncio.run(main())

Tag summary

Content type

Image

Digest

sha256:ef3031597

Size

117.9 MB

Last updated

7 months ago

docker pull affinefoundation/lgc:pi-v2