AI Agent Development

In this section, we review existing studies related to the challenges and concerns in the adoption of MCP. Existing studies have explored these challenges [3, 5, 7, 10, 13] only from security and maintainability perspectives. Hou et al. [7] regard MCP as an open standard that specifies a unified, bi-directional communication and dynamic discovery protocol between Al models and external tools or resources that enhance interoperability and reduce fragmentation across diverse systems. They focus on security concerns by creating a threat taxonomy that comprises four major attack types: malicious developers, external attackers, malicious users, and security flaws. To validate these risks, the authors develop and analyze real-world cases that demonstrate attack and vulnerability types with MCP implementations and propose a set of actionable steps for a secure MCP adoption.

Hasan et al. [5] perform an empirical study of official and community MCP servers. Focusing on health metrics and a hybrid analysis pipeline, the authors evaluate 1,899 open-source MCP servers, using static code analysis tools. The authors identify eight vulnerability types - credential exposure, lack of access control, CORS issues, improper resource management, transport security issues, authenti-cation issues, insecure file creation, and input validation issues. The authors also evaluate how healthy and sustainable MCP servers are, using 14 repository-level metrics from existing literature, and find that they demonstrate higher or equal median values in 9 out of 14 key metrics in comparison to OSS and ML baselines, concluding on promising sustainability.

More recently, Tiwari et al. [13] perform an audit of MCP in vision systems, identifying weaknesses in schema semantics, in-teroperability, and runtime coordination by analyzing 91 publicly registered vision-related MCP servers to identify failures and cate-gorize them. The authors define the following failure modes: schema divergence and interface ambiguity, lack of runtime schema valida-tion, composition failures and bridging scripts, and the absence of evaluation frameworks.

Existing research has primarily focused on static code analysis to identify issues such as security vulnerabilities and maintainability issues; our research focuses on self-reported challenges by devel-opers in the adoption of MCP servers. We go beyond theoretical analysis to gain insight directly from developer reports. Hence, pro-viding a structured categorization of issue types, resolution insights, and recommendations on how to mitigate these challenges.

Prior Research vs. Our Study on MCP Challenges

Our goal is to identify the challenges faced by developers in the integration of MCP and evaluate the level of work required in resolving them. In this section, we describe the dataset used in our study. We present our approach for the dataset curation in Figure 1.

3.1 Dataset Collection and Filtering

To understand the challenges developers face when integrating MCP servers, we analyze issues from the official MCP servers repository [1]. We focus exclusively on the seven reference servers. Table 1 presents the description of each of the reference servers. We focus on the reference servers because these were built by Anthropic, the protocol's authors, and they therefore adhere closely to the offi-cial specification. Limiting our scope to them reduces noise from third-party misinterpretation of the specification, non-adherence to the specification, or poor code quality, and provides a consistent baseline for assessing the protocol's maturity.

Using the GitHub API, we collect all 684 issues in the MCP official GitHub repository as of October 10, 2025. We filter the issues labeled "bug", yielding 309 issues. We include both open and closed issues, as our goal is to categorize all issues encountered in the integration of MCP, regardless of whether they have been fixed or not.

To ensure that our dataset contains only issues related to ref-erence servers, we apply a keyword-based filter implemented in Python that searches for specific substrings associated with ref-erence servers as well as patterns that frequently appear in MCP server installation. Our filtering approach assumes that any issue without a mention of a reference server-related term or key-word (e.g., git-mcp, fileserver, filesystem-mcp) falls outside the scope of our research. We employ keyword-based filtering be-cause it is widely used in empirical software development research, and it provides an efficient and reproducible approach to cleaning data [11, 12]. For example, to identify issues relating to the Fetch MCP server, we use keywords such as 'fetch-mcp', 'fetch mcp', 'fetch-server', 'fetch server', 'mcp fetch', 'mcp-fetch', 'server/fetch', 'server fetch', 'server-fetch', 'mcp/fetch', 'mcp-server-fetch', and 'mcp_server_fetch'. For an MCP server like Fetch, we intentionally avoid using the standalone keyword "fetch", as it could appear in unrelated contexts (e.g., Javascript fetch() calls). Similarly, for the Git reference server, we carefully construct our keywords by includ-ing spaces after the keyword "git " to exclude third-party servers like GitHub and GitLab, which are outside the scope of this study. After applying our filters, our dataset was reduced to a total of 153 issues.

3.2 Dataset Validation

To confirm the validity of our automated filtering, the first two authors independently conduct a manual review of all 153 issues to filter out any leftover non-reference servers. Any discrepancies and disagreements in the filtering are resolved through discussion. After this process, we identify 17 false positives - issues incorrectly classified by the script as reference server-related and filter them out. These false positives account for approximately 11% of the dataset. The manual validation process is to ensure that our final dataset of 136 issues accurately reflects reference server issues.

MCP Reference Servers Overview

Our goal is to identify the challenges faced by developers in the integration of MCP and evaluate the level of work required in resolving them. In this section, we describe the dataset used in our study. We present our approach for the dataset curation in Figure 1.

RQ1: What are the common challenges in the integration of MCP?

Despite MCP gaining popularity in the standardiza-tion of how LLMs interact with external data sources and tools [5], there exists no empirical data of developer-reported challenges in its adoption. Existing studies only focus on security vulnerabilities and static code analysis [5, 7]. Thus, in this RQ, we aim to identify and categorize these challenges as reported by developers. This will provide insights to the repository maintainers and contributors on the different prevalent issues reported by developers. Hence, guiding future improvements and reducing the challenges faced by developers.

File Path and File System Issues (21.32%): These are issues per-taining to file system operations across various operating systems. Our analysis shows that this category accounts for the most re-ported issues by developers. A significant number of these issues involve path conversion and resolution across different operating systems, usually resulting in ENOENT error (file not found) or access denied errors (e.g. issue #2526). Several developers re-port cases where valid file paths are incorrectly resolved due to case sensitivity inconsistencies, relative path handling, or path valida-tion (e.g. issue #1987). Other file system issues reported demonstrate problems encountered when file paths contain spaces, hyphens, or special characters, often resulting in failures (e.g issue #1770). Additionally, developers report failures related to the creation of directories and access control. For example, there are reports about the inability to recursively create directories (e.g issue #711), access-denied errors for legitimate paths (e.g issue #470), and unexpected access to files outside a sandbox environment via symlinks (e.g. issue #1845), indicating a potential security vulnerability. Notably, there are reports of complete server crashes when configured paths are invalid (e.g. issue #2815), suggesting inadequate error handling logic. In summary, these findings show the need to ensure reliable and secure cross-platform file system operations. It is worth noting that the majority of these issues are specific to the FileSystem MCP server, suggesting that these challenges are primarily from the server's implementation rather than the protocol itself.

Data validation and Type Issues (13.97%): These are issues involving incorrect data types, inadequate input validation, and wrong type checking. They usually surface after successful server initialization. Specifically, a significant number of these issues per-tain to schema compliance failures. For example, in issue #2473, the Sequential Thinking server frequently fails with “invalid to-talThoughts: must be a number” when clients pass non-numeric values or improperly formatted parameters. Similarly, there are reports of parameter mismatches. Another case in this category is where Claude passes “undefined” instead of a properly structured object when making server calls (issue #2410), resulting in “invalid arguments” validation errors. Last in this category are reports per-taining to character encoding which particularly affects Windows users, where servers fail to handle non-ASCII characters properly (e.g. issue #2098).

Tool Invocation, Visibility, and Behavior Issues (13.24%): This category captures issues with tool discovery, invocation, and un-expected behavior during tool execution. Specifically, we capture issues relating to difficulty with Al agents identifying the right tool to use. There exists a report in issue #2818 about Everything server, where the client requests available tools using the standard tools/list method, but fails with “unable to read roots on an unde-fined property”, suggesting an incomplete error handling in the tool listing process. There is also a persistent JSON parsing error in the Memory Server’s “create_entities” and “create_relations” tools (issue #2689), indicating problems with data serialization. There is also a severe problem where the “git_add” tool incorrectly tracks “./git/” directory files, causing repository corruption (issue #2373).

Configuration and Environment Issues (12.50%): The configu-ration and environment category covers problems with setup and configuration parameters. Unlike the startup failures that prevent servers from launching, these issues involve servers that are able to start but are unable to operate correctly due to configuration problems or environment setup issues. Some developers report corruption to local environment files. Notable in this subset is a situation where a developer reports the Memory server’s “mem-ory.json” file getting corrupted due to concurrent write operations, resulting in JSON parsing errors (issue #2579). This corruption occurs when multiple processes attempt to modify the configura-tion file simultaneously, resulting in file corruption, which then requires manual update. Another developer reports an unexpected functionality caused by an out-of-date package. (issue #2431). In this scenario, the affected server lists fewer tools than documented, causing a confusion with understanding what the correct list is. Fur-thermore, there exists a report of a tool availability discrepancy due to a server being published under an incorrect label (issue #1908).

Documentation and Tool Description Gaps (11.76%): This cate-gory of issues represents usage confusion as a result of inconsistent, inaccurate, or missing documentation or tool descriptions. For ex-ample, in issue #2032, a user reports that the “read_multiple_files” tool of the FileSystem server lacks a clear description, causing AI assistants to use it incorrectly. Similarly, Al systems confuse the use of “search ”search_files” and “search_file_contents” tools due to over-lapping descriptions (issue #896). Also, there are reports of tools with missing documentation of optional parameters, default values, or parameter constraints, forcing users to resort to trial-and-error approaches. Last in this category are reports of broken and non-existent reference links in the repository’s documentation (e.g. issue #294).

Connection and Communication Issues (11.03%): This category of issues captures reports relating to unstable or failed connections between MCP clients and servers. Developers report sudden dis-connections or unexpected termination of connection after the initialization phase of the protocol. When this situation occurs, the MCP server unexpectedly exits instead of maintaining a persistent connection (e.g. #issue 2517). Common error messages reported are “Not Connected” and “Connection closed”. Some reports also attribute these challenges with incompatibilities across various AI clients. For example, a developer reports in issue #2338 about con-nection problems with Claude. In summary, this category highlights the challenge of maintaining a stable client-server connection.

Start up and Initialization Failures (7.35%): These are issues that prevent the MCP server from moving past the initialization phase. They are critical issues, as they render the MCP server com-pletely unusable. A significant number of these issues are a result of missing or incompatible dependencies. For example, a devel-oper reports in issue #2421 that the Sequential Thinking MCP server fails to start in Cursor IDE with the error “Cannot find module...node_modules/zod/index.js”. Another developer reports successfully building a Dockerfile to run the Fetch MCP server but unable to get a response from the initialization phase. Notable in this category is a report where calls to the Sequential Thinking server frequently hangs indefinitely (issue #2762). All of these is-sues highlight the critical nature of this category as it often makes it impossible to use the affected servers.

Timezone Issues (2.94%): This category of issues pertains to time zone handling and localization problems, particularly affecting the Time MCP server. These issues highlight challenges supporting users across different geographical locations, languages, and day-light saving time transitions. For example, in issue #786, a user reports that the server fails during daylight saving time transitions from EST to EDT, resulting in an error “No time zone found with key EDT”. Additionally, a significant barrier is observed for inter-national users with timezone names where the server is unable to map non-English timezone names to standard ones. For example, in issue #231, the reporter complains that the server fails when the time zone name is not in English.

Crashes and Runtime Errors (2.21%): For this category, we assign issues where there are reports of abrupt crashes, usually without an explicit or clear error message. For example, in issue #2792, a user reports that the Sequential Thinking server sometimes crashes, producing random errors even when such operations worked pre-viously. This type of issues raises concerns about the reliability of these servers for production use cases.

Build Errors (1.47%): These are challenges that occur when devel-opers attempt to build their application in which an MCP server is integrated. Specifically, they occur as a result of dependency issues in specific MCP servers. For example, a user reports getting miss-ing dependency errors after running “pnpm run -r build” in issue #1589. Another user reports a Docker build failure due to version inconsistency in “uv.lock” in issue #997. While it is unclear if these are issues with the implementation approach by the user or MCP itself, it is worth stating for completeness of our result.

Others (2.22%): We categorize three issue types as “others” since they appear only once, each in our dataset. They include progress notification, missing resource, and insufficient logs issues. The first is the progress notification issue as reported in #2621. MCP’s progress notification mechanism is designed to provide real-time updates for long-running tasks. This user reports that while the progress notifications are successfully generated and visible in the browser network tab, the MCP inspector does not react to them until the tool ends with a final response. Additionally, in issue #625, a user reports that the “resources/list” method implementation is missing while the other methods work as expected. Lastly, a user complains in issue #106 about the need to have developer logs for failing MCP server invocations, as the displayed logs are not helpful.

Developer-Reported Issue Categories (RQ1)

RQ2: What is the extent of work involved in resolving issues affecting MCP server integration?

Motivation. In this RQ, our goal is to understand the extent of work involved in resolving the issues identified in RQ1. The official MCP repository currently provides no visible mechanism to prioritize or rank issues. The aim of this RQ is to measure the level of work required to resolve the various categories of issues by analyzing the pull requests that resolved the issues. With this, the maintainers of the MCP repository will be able to identify the areas of the repository that require more attention and prioritize them accordingly.

Approach. To answer this question, we adopt the use of code churn to measure the extent of work required to resolve the re-ported issues. Code churn has been widely adopted in empirical software engineering research as a metric for measuring devel-oper effort involved in code changes [4]. For the purpose of this research, we define code churn as the sum of the lines of code added and deleted. We filter all closed issues from our dataset. Issues are closed for various reasons, including those that could not be re-produced by the maintainers. To focus only on issues that were closed after they were successfully resolved, we use the GitHub API to identify pull requests that are linked to those issues. First, we perform a search for all pull requests where each issue in our dataset was mentioned using GitHub’s search API using the query “repo:modelcontextprotocol/servers type:pr #issue_number”. Sec-ond, using the GitHub’s timeline API, we mine all “closed” events linked with each issue and cross-referenced PR links to identify issues that were resolved through merged pull requests. We iden-tify that some issues were resolved by multiple PRs and factor that in our analysis. We observe that some PRs were explicitly men-tioned in issue discussions as the solution to such issues without being directly linked. To capture such PRs, we implement regular expression patterns to identify such references in the discussion.

Next, we sum up the total number of lines of code added and the total number of lines deleted in the PR linked to each issue. Since some issues are resolved with multiple PRs, we sum up the metrics in all PRs associated with such issues. So, for example, if an issue was confirmed resolved after four PRs were merged, we sum up the metrics in all associated PRs. Then, we aggregate the metrics per issue category to calculate the average churn. Out of all the issues in our dataset, 80 of them are closed, out of which 41 have associated PRs.

Results. Table 3 presents the results of our analysis. We only present results for categories that have at least five resolved issues. Our analysis reveals that connection and communication issues require the highest amount of work to resolve (Avg churn: 1143.5). Trailing that is the File Path and File System category (Avg. churn: 492.0), followed by Data Validation and Type Issues (Avg. churn: 402.7) and Documentation and Tool Description Issues (Avg. churn: 309.6). Furthermore, Configuration and Environment Issues and Tool Invocation, Visibility, and Behaviour Issues have average churn values of 143.5 and 95.7 respectively.

Code Churn Across Issue Categories (RQ2)

We interpret our results and provide recommendations for MCP repository maintainers and contributors to guide them on how to mitigate these challenges.

Examining the Most Prevalent Challenges.

The result in RQ1 shows that file path and file system issues are the most prevalent ones (21.32%). Notably, this category highlights some security vulnerability concerns, as shown in issue #1845 where an Al agent is able to access files outside a clearly defined sandbox environment via symbolic links. This aligns with findings from Hasan et al. [5], who mentioned insecure file creation as one of the eight vulnera-bility patterns identified in MCP servers. Our work extends this by showing that these issues go beyond theoretical speculation and rep-resent practical challenges developers face. Data validation and type issues (13.97%) expands on another finding of Hasan et al. [5] who listed input validation issues as another vulnerability pattern identified in MCP servers. Similarly, Tiwari et al. [13] identified a lack of runtime schema validation as one of the flaws identified in their study. Tool invocation, visibility, and behaviour issues (13.24%) are concerning because they impact the core idea of MCP, which is to have a seamless tool integration with LLMs.

We recommend a comprehensive review of the filesystem server to address the highlighted challenges. Also, we recommend the development of a reusable input validation library that is well-suited for the MCP ecosystem. Additionally, the MCP ecosystem would benefit from comprehensive tests across multiple real-world scenarios to mitigate the tool-call challenges.

The Need for a Prioritization Strategy.

Our analysis in RQ2 shows a concerning detail about the resolution rate. While tool in-vocation is central to the core idea of MCP, only 27.78% of the issues have been resolved. Similarly, only 30% of start up issues have been resolved. This is concerning since such issues completely render an MCP server unusable, as they occur during the initialization phase of the MCP lifecycle³. We recommend that the repository main-tainers implement a prioritization strategy based on these findings. Additionally, tool description issues are critical as wrong tool calls can be catastrophic, potentially causing irreversible damage.

Threats to Construct Validity consider the extent to which our analysis accurately reflects the theoretical constructs it is intended to assess. A threat to construct validity is our reliance on developer-reported GitHub issues, which may not fully represent all the chal-lenges experienced by developers in practice. However, Github issues are a widely used and accepted data source in empirical software engineering research.

Threats to Internal Validity consider the researchers’ bias and errors. First, our method of analysis is primarily manual. The subjec-tive nature of manually categorizing the issues is a potential threat to the validity of our study. This subjectivity may affect the repro-ducibility of our categorization. To mitigate this risk, two authors independently categorized the GitHub issues and calculated the Cohen’s Kappa coefficient, achieving a substantial agreement value of 0.68. Discrepancies and disagreements were resolved through discussion among the authors. Also, in our GitHub issues dataset, we applied a Python script to filter out non-reference MCP servers using keyword-based filtering. However, there is no assurance that all issues captured are indeed reference servers only. To mitigate this, two authors independently reviewed each issue in our dataset manually and discussed disagreements to reach a consensus. Addi-tionally, percentage of issues resolved may not be a great metric as it is time-dependent. Hence, we focused on measuring code churn as the metric for evaluating the extent of work in resolving issues.

Threats to External Validity consider the generalizability of findings. In our study, we focused only on reference servers, which may not fully represent the wider range of MCP servers. This ap-proach was implemented to mitigate the introduction of variations that may arise from third-party developer integration methods. Fu-ture work can go further to analyze community-developed MCP servers.

In this study, we presented a comprehensive analysis of developer-reported MCP issues, categorized them, and highlighted the most prevalent ones. Through manual inspection of the 136 reference server-related issues from the official MCP repository, we identified significant patterns in the issue types, we proposed resolution strate-gies, and evaluated the extent of work required to resolve each issue category using average code churn. Our findings revealed the need to prioritize file path and file system issues, tool invocation-related issues, start up and initialization issues, and documentation/tool description issues. While file system issues are the most common, communication issues require the most effort to resolve. Based on our findings, we made several recommendations. Notably, the need to develop an issue resolution prioritization strategy, as our findings show that the most critical issues have very low resolution rates. Our study provides the foundation to develop such a prioritization strategy. Future work can compare these findings with community servers to identify similarities and differences in issue types and their prevalence.