Skip to content
View feiiiiii5's full-sized avatar
🏠
Working from home
🏠
Working from home

Block or report feiiiiii5

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
feiiiiii5/README.md

Yufeiyang Chen

Undergraduate in Cyber Science and Technology at Sun Yat-sen University. I have been contributing to open-source AI tooling since July 2026 and still send patches most days, mostly to evaluation and red-teaming frameworks, agent and MCP tooling, and the libraries they sit on.

Much of what I fix has the same shape. Something fails, nothing crashes, and the caller gets a value that looks like success. A failed page read comes back as if it were the page; a scanner reports that it wrote its findings when the write never happened.

Where I contribute

Area Projects
Evaluation and red teaming PyRIT · inspect_ai · inspect_evals · lm-evaluation-harness · opik · uqlm · trulens · rhesis · garak · lmms-eval
Agents, coding agents and MCP MCP servers · fastmcp · serena · letta-code · pydantic-ai · livekit agents · haystack · llama_index · griptape · fantasy · mcp-context-forge · zotero-mcp
Structured generation and inference xgrammar · outlines · vllm-metal
Tracing and observability openinference · phoenix · openlit · langwatch
Security tooling fickling · AI-Infra-Guard · agentic_security · agent-sweep
Data validation and ingestion pandera · great_expectations · qdrant-client · unstructured

Some pull requests I learned from

  • inspect_evals #2132. With more than one epoch, Humanity's Last Exam averaged each sample across epochs before computing calibration error, so errors in opposite directions cancelled and a maximally miscalibrated run scored 0.0. Calibration is now computed per attempt, and single-epoch results are unchanged. The issue was opened by a maintainer.
  • PyRIT #2467. Gave the GCG optimisation loop explicit state types. In review the maintainer caught that my first version used inf to mean "not measured", which would have hidden a genuine non-finite loss; it now carries an explicit flag. The issue was opened by a maintainer.
  • uqlm #459. When one judge in a panel ran out of retries, every aggregate for that prompt became NaN and the run still reported success. I reported it and sent the fix together.
  • lm-evaluation-harness #4039. MATH answer normalization turned tuple answers such as 0,1 into 01, so correct answers were marked wrong. The maintainer kept the leaderboard copy of the function frozen, because changing it would make historical scores incomparable. I had not thought of that, and it was the right call.
  • opik #8195. A system under evaluation could embed a verdict in its output that the judge would repeat, and the parser took the first one. My first revision escaped the values; the maintainer pointed out that rewriting the evaluated output distorts the evaluation itself, so the merged version isolates the output with namespaced delimiters instead and states the remaining risk in tests. The issue was reported by another contributor.
  • xgrammar #834. Made prefixItems positional, as JSON Schema Draft 2020-12 specifies. An earlier draft accepted too much; the regression matrix caught it and it was replaced.

failroute

failroute is a static analyzer I wrote for one family of these bugs: Python exception handlers that return something a caller cannot tell apart from success. pip install failroute.

Measuring it was the most useful part. On eight pinned AI packages it flags plenty that standard linters miss, but in a sample labelled by LLM agents most findings were intentional fallbacks, and among findings the linters miss, about 1 in 56 was labelled a defect. The defects it did find are also caught by flake8 --select E722. The repository has the full numbers and the method.

How I work

Before filing anything I follow the call path in the real code. When I ran failroute over PyRIT it raised several dozen warnings; I traced twelve of them, all twelve were deliberate design decisions, and I filed nothing.

I work with coding agents inside a workflow I set up myself, with my own test gates, and pull requests where AI tools shaped the change say so. Early on I gave the agents too much rope: they lost context over long runs and opened some weak and duplicate pull requests. I closed those and tightened the workflow. If one of my pull requests turns out to be wrong or not worth your time, I close it; if you find one I missed, close it or tell me. Blunt review is welcome.

Most repositories on this account are forks for upstream work.

Pinned Loading

  1. failroute failroute Public

    Static detection of failure-routing anti-patterns in Python (swallowed exceptions, silent fallback returns, masked exceptions), with an empirical study of how often findings are real bugs. AST-base…

    Python 2

  2. comet-ml/opik comet-ml/opik Public

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Python 22.2k 1.8k

  3. EleutherAI/lm-evaluation-harness EleutherAI/lm-evaluation-harness Public

    A framework for few-shot evaluation of language models.

    Python 14.1k 3.6k

  4. microsoft/PyRIT microsoft/PyRIT Public

    The Python Risk Identification Tool for generative AI (PyRIT) is an open source framework built to empower security professionals and engineers to proactively identify risks in generative AI systems.

    Python 4.5k 910

  5. mlc-ai/xgrammar mlc-ai/xgrammar Public

    Fast, Flexible and Portable Structured Generation

    C++ 1.9k 217

  6. UKGovernmentBEIS/inspect_evals UKGovernmentBEIS/inspect_evals Public

    Collection of evals for Inspect AI

    Python 683 445