Skip to content

[api][integration] Support agent skills in python#596

Open
wenjin272 wants to merge 4 commits intoapache:mainfrom
wenjin272:skills-python
Open

[api][integration] Support agent skills in python#596
wenjin272 wants to merge 4 commits intoapache:mainfrom
wenjin272:skills-python

Conversation

@wenjin272
Copy link
Copy Markdown
Collaborator

@wenjin272 wenjin272 commented Mar 26, 2026

Linked issue: #590

Purpose of change

This PR provides basic support for using agent skills in the Flink Agents Python API. We will further enhance usability in subsequent PRs.

Tests

ut & e2e

API

Yes, add abstraction for agnet skill, and modify the chat model api.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

@github-actions github-actions bot added doc-needed Your PR changes impact docs. fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue. and removed doc-needed Your PR changes impact docs. labels Mar 26, 2026
@wenjin272
Copy link
Copy Markdown
Collaborator Author

Regarding the skill e2e testing, I evaluated qwen3:8b, qwen3.5:4b, and qwen3.5:9b. I found that only qwen3.5:9b with thinking enabled could consistently follow the prompts. However, in the GitHub CI environment, the lack of GPU resources results in very slow model inference, causing tests to take over 30 minutes to complete. Consequently, I have temporarily skipped the skill e2e tests.

I plan to dedicate time later to investigate and resolve this issue.

@wenjin272 wenjin272 changed the title Skills python Support agent skills in python Apr 1, 2026
@wenjin272 wenjin272 changed the title Support agent skills in python [api][integration] Support agent skills in python Apr 1, 2026
@github-actions github-actions bot added doc-needed Your PR changes impact docs. and removed doc-needed Your PR changes impact docs. labels Apr 1, 2026
Copy link
Copy Markdown
Collaborator

@alnzng alnzng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments, will continue the review. BTW, for future PRs, I'd suggest separating code formatting / refactoring changes from actual logic changes into separate PRs.

)


# TODO: Implement
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to remove this unimplemented class to avoid confusion.


@staticmethod
def _split_commands(command: str) -> List[str]:
"""Split a command string by shell operators, respecting quotes.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, The intention of this method is to extract individual executable commands from a compound command string, not to perform a full shell/bash tokenization. Shell has many operators (e.g. redirection >, >>, <), but this method only targets command-separating operators, those that connect multiple executables. Can you confirm? If yes, let's make it clear in the comment.

Also can we confirm all command-separating operators are covered? For example, are && and || handled correctly?

Out of curiosity, is there any native Python lib or thirdparty lib could provide this kind of functionality? I am not sure if we covered all the necessary shell / bash operators, so I'm thinking if there is a such kind of lib then we could just use it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the functionality of this function is indeed as you described.

I could not find an existing third-party library that provides this functionality, so I implemented one myself by referencing agentscope-java.

I added some tests to confirm that delimiters such as &&, |, and ; are handled correctly. Additional tests are needed for ||.

Actually, I currently feel that the security validation for our execute_shell_command might be implemented too simply. I mainly referenced agentscope-java, but the bash tool in Claude Code's implementation includes extensive security validation rules. Therefore, I may refactor it by referencing Claude Code.


if skill.name != skill_dir.name:
logger.warning(
f"The skill name {skill.name}is different from the base directory {skill_dir.name}."
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{skill.name}is -> {skill.name} is

)

# Pattern to match key: value format
KEY_VALUE_PATTERN = re.compile(r"^([a-zA-Z_][a-zA-Z0-9_-]*)\s*:\s*(.*)$")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we can remove this? I didn't see it is used in any other place.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, the coding agent generated code that used regular expressions to parse YAML frontmatter. I later switched to using a YAML library but forgot to remove these two PATTERNs.
I will remove this.

return ParsedMarkdown(metadata={}, content=markdown_content)

try:
metadata = yaml.safe_load(yaml_content)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does markdown frontmatter always follow YAML syntax?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the Agent Skills specification: https://agentskills.io/specification#skill-md-format, The SKILL.md file must contain YAML frontmatter followed by Markdown content.

class MarkdownSkillParser:
"""Utility for parsing Markdown files with YAML frontmatter."""

# Pattern to match frontmatter: starts with ---, ends with ---
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is below a valid frontmatter?

 ---
  name: tricky
  description: |
    This has a
    --- inside
  ---

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a valid frontmatter, and the parsed result is {'description': 'This has a', 'name': 'tricky'}, the left content will be regarded as markdown content inside\n---\n.

and optional resource files:

baseDir/
├── skill-name-1/
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if the users define folder with other different names? For example, this Claude official skill: https://github.com/anthropics/skills/tree/main/skills/skill-creator

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this question. Regarding skill-creator, then the baseDir can be any dir the flink-agents job can access, like /flink/usrlib/skills, and the skill-name-1 will be skill-creator.

return func


def skills(func: Callable) -> Callable:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong commit

"so we will not fix this issue for now."
)
def test_long_term_memory_async_execution_in_action(tmp_path: Path) -> None: # noqa: D103
def test_long_term_memory_async_execution_in_action(tmp_path: Path) -> None:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong commit

Only the first token of a command is checked against this list.
"""

paths: List[str] = Field(default_factory=list)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest to provide some factory methods, like Skills.from_local_dir() or Skills.from_url(). This aligns with Prompt.from_text() / from_messages(), and is more easy-to-use.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And maybe hide the implementation of Skills from the users.

"""Get another resource declared in the same Agent."""

@abstractmethod
def generate_skill_discovery_prompt(self, *skill_names: str) -> str:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest the name generate_available_skills_prompt

VECTOR_STORE = "vector_store"
PROMPT = "prompt"
MCP_SERVER = "mcp_server"
SKILL = "skill"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SKILLS

TOOL("tool"),
MCP_SERVER("mcp_server");
MCP_SERVER("mcp_server"),
SKILL("skill");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SKILLS

from flink_agents.runtime.skill.skill_repository import SkillRepository


class RegisteredSkill:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need both AgentSkill and RegisteredSkill ?

Comment on lines +69 to +72
allowed_script_types: List[str] = Field(
default_factory=lambda: ["shell", "python"]
)
allowed_commands: List[str] = Field(default_factory=list)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't sounds right. How is allowed script types / commands an attribute of the skills? If the model decide to execute something, how do we know it's following the instructions of a skill or not?

class ExecuteCommandArgs(BaseModel):
"""Arguments for ExecuteCommandTool."""

skill_name: str = Field(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about requiring a skill name for executing a command. Skills are not executable programs like tools, but are a set of knowledges, scripts and resource for doing something. It's the LLM/agent that learns a skill and perform actions. You cannot execute a skill, nor saying an action belongs to a certain skill.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-needed Your PR changes impact docs. fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants