Llm-Powered Multi-Agent System for Automated Data Preprocessing

Chitran Chakraborty

Audio by Paper2Audio.

1. Introduction

Data preprocessing forms the first step in the machine learning process; the nature of input data determines the performance of models in terms of both efficiency and reliability. Data is seldom preprocessed before it is analyzed. The real world data may have gaps, duplicates, outliers, incorrect categorical values, skewed distribution, irrelevant attributes, high dimensionality of feature space, and imbalanced target classes. Data preprocessing includes de-tection of these properties of the specific data set and proper preprocessing techniques for that learning problem.

Traditional preprocessing pipelines are always developed as statically defined or manually engineered pipelines. A data scientist must analyze the dataset and recognize its possible data quality issues, select appropriate methods and determine the order in which they will be used. It is relatively easy to do when working with known datasets, but becomes increasingly difficult to achieve in terms of generalization and maintenance as the number of structural, type, quality problems and learning objectives starts to vary for different datasets – the transformation that might work well with one dataset might not be necessary or even damaging with the other.

Large language models have brought about new opportunities for automating data-related reasoning processes – the ability to understand structured data, the ability to reason about potential pre-processing methods, as well as communication with computing devices. However, delegating all preprocessing to a general-purpose agent complicates the modularity and scalability of such systems. In fact, pre-processing of data involves a number of different processes – missing value imputation, outlier removal, categorical variables encoding, numeric transformations, feature selection, etcetera – each of which has its goals, constraints, and criteria.

Multi-agent systems provide an architecture whereby complex processing tasks are decomposed among various agents that are specialized in their own areas. The idea is that instead of a single agent doing everything regarding the preprocessing tasks, different agents can specialize in the type of operations to be done to improve data quality and other preprocessing techniques. This allows you to have a modular system wherein various capabilities regarding preprocessing can be created and improved independently.

This paper proposes an L.L.M-based multi-agent system for automated pre-processing of C.S.V data. LangGraph serves as a means for coordinating the actions of specialized preprocessing agents via the shared state mechanism. The orchestration module starts by analyzing the features of the input dataset as well as the learning objective, and uses an L.L.M to determine the required pre-processing agents. The determined agents are then able to perform dataset profiling, missing value imputation, outliers detection, categorical encoding, numerical transformation, dimensionality reduction, and class balancing. The sequential operation of agents implies that the preprocessed dataset together with any generated intermediate knowledge is passed on to the next stage.

Besides creating a preprocessed data set, the system creates a variety of outputs to capture and replicate the preprocessing workflow, including the preprocessing report, the executable Python code for preprocessing, and a Jupyter notebook. The system creates a quality score and also offers a browser interface for agent execution and reviewing preprocessing output.

The primary goal of the current research is to examine the hypothesis that a particular multi-agent architecture presents a suitable way to perform automated data preprocessing. In contrast to a static pipeline, the suggested solution attempts to adaptively choose the preprocessing steps in accordance with the features of the used dataset and a set learning goal. The analysis of the performance of the system is performed by comparison with appropriate baselines for various datasets, quality levels of data and measures.

The key contributions of this study are as follows:

- We advocate an architectural approach for automated data preprocessing, where different agents are designed to solve a different class of data quality and feature processing problems.

- We propose an orchestration framework that makes use of an L.L.M to identify the properties of the dataset and then identify which preprocessing agents are required.

- The combination of several preprocessing operations, including the treatment of missing values, outliers, categoricals, numerals, dimensionality, and class imbalance, is accomplished using LangGraph's stateful workflow.

- We create preprocessing artifacts that include the preprocessing report, the Python code used for preprocessing, and the Jupyter Notebook.

- We conduct an experimental study of the efficiency, cost, and drawbacks of the presented multi-agent framework, in comparison to suitable preprocessing approaches.

The rest of this paper is structured as follows. Section 2 surveys the current literature in the domains of automated data cleaning, data preprocessing, data processing by L.L.M's, and multi-agent systems. Section 3 describes the architecture and functioning of the system proposed in this paper. Section 4 describes the datasets, baseline methods, evaluation criteria, and methodology of experiments. Section 5 describes the results of experiments and comparative analysis.

Section 6 analyzes the results obtained and their limitations and implications. Section 7 concludes the paper and proposes areas for future research.

2. Related Work

Data Preprocessing Automation has been thoroughly investigated with respect to Data Quality Management, Automated Machine Learning, and Intelligent Data Preparation. Existing solutions include rule-based data cleaning techniques, automated machine learning workflows, and more recently, language model based systems for data reasoning and tool utilization. This section presents an overview of these research efforts and places the proposed multi-agent preprocessing system in this wider perspective.

A. Data Cleaning Automation and Data Quality

In the past, data cleaning has concentrated on the identification and removal of errors to maintain the quality of the analytical process or machine learning process. The issues usually include missing values, duplication of data, inconsistencies, constraint violations, and faulty numerical measurements. The early studies confirmed that data cleaning is an integral part of data management using methods based on pre-defined rules, integrity constraints, statistics, and domain knowledge.

The systems like OpenRefine have demonstrated the usefulness of interactive data cleaning, as they provide tools to transform and reconcile heterogeneous data sources. The automated data cleaning literature has looked into various methods for identifying and fixing errors by means of statistical and constraint-based approaches. While such methods have been shown to work reliably when the assumptions about data and data cleaning are known beforehand, they are highly rule-dependent.

This becomes even more significant where data sets vary in structure and contain more than one interrelated problem to deal with. What fits one data set might be inappropriate for another, depending on the distribution and nature of features and the goals of learning. This has led to an increased interest in automatic techniques that learn or infer preprocessing actions.

B. Machine Learning Automated and Data Preparation

AutoML systems aim to automate the process of creating machine learning pipelines, thereby reducing the amount of manual work involved. Some of the AutoML systems that have been built include Auto-weka, tpot, Auto-sklearn, and more recently, AutoML frameworks that automate some portion of the model and pipeline selection process.

The example of AutoML frameworks is indicative of how data preparation processes could be considered as a problem of searching for pipelines automatically. Yet, it is important to highlight the fact that most frameworks use a pre-defined space of search and apply systematic optimization techniques as opposed to linguistic reasoning regarding the dataset.

The system is based on a different principle. Instead of doing a full search in the predefined space of the pipeline, an L.L.M-based orchestrator analyses the properties of the dataset and chooses special-purpose preprocessing agents accordingly. This allows describing the preprocessing policy using a modular agent architecture, but still implementing the data transformation using conventional programming techniques.

C. Large Language Models for Data Science

Recent developments in large language model capabilities have led to the emergence of novel techniques for data analysis and data preparation. Large language models are able to understand natural language descriptions of data-related tasks, perform reasoning on structured information, produce code that can be executed, and interface with external software. It has therefore led to exploration of methods using language models as interfaces to data science workflows.

Recent advancements in the abilities of large language models have brought about new methods for data analysis and data pre-processing. Large language models are able to understand instructions provided in natural language, think about structured data, write codes, and communicate with external systems. This has inspired researchers to develop techniques for utilizing language models as interfaces for data science applications.

However, there are certain issues associated with L.L.M's too. There is a possibility that the output provided by them might be inappropriately transformed, contain wrong code or wrong decisions which cannot be easily validated in automatic mode. Heterogeneous tasks of pre-processing assigned to one model result in monolithism of a process when each step cannot be easily evaluated.

D. L.L.M-based Agents and Multi-Agent Systems

New studies have considered the employment of agents based on language models for complicated processes involving planning, tool manipulation, task composition, and collaboration among several specialized modules. Frameworks for agents enable language models to act as decision-making modules capable of invoking external tools and maintaining their state across several iterations.

Multi-agent architectures take this approach further by dividing up the complicated job among several agents, each with its own special purpose. In multi-agent systems, the agents could coordinate via state sharing, message passing, or orchestration. Dividing the complicated process into smaller pieces will help relieve any heavy processing that is being done by one single agent.

It will be especially useful when applied to automated data preprocessing. Instead of employing a universal component to process missing values, outliers, categorical features, numerical conversions, and imbalances among classes, each of these processes can be assigned to its dedicated agent and then managed by a controlling process that determines which particular agents are applicable for a given data set.

E. Research Gap

Some of the important groundwork that the proposed system builds upon is found in the existing literature. Classical data-cleaning systems offer proven methods of discovering and fixing data quality issues, while AutoML systems show that it is possible to automate the data preprocessing steps. Recent systems based on L.L.M's incorporate the benefits of flexible reasoning and coding, while the field of multi-agent systems offers architectural solutions for task decomposition.

Nevertheless, there is a number of issues that remain under- explored in the field of automated end-to-end data preprocess- ing. For one thing, existing data-cleaning systems usually rely on pre-defined rules or specific configurations, thus failing to adapt themselves to new data properties. Secondly, AutoML techniques are based on the concept of searching for the most optimal pipeline in the set of predefined transformations.

Thirdly, L.L.M data science systems provide flexible reasoning yet fail to decompose the process of data preprocessing and validate each transformation. Lastly, the multi-agent paradigm has proven its efficiency in general data science and reasoning but the effectiveness of decomposing agents for heterogeneous data preprocessing operations remains unknown.

This paper is concerned with a multi-agent architecture that is tailor-made for automated data preprocessing. In this proposed solution, decision-making via L.L.M is supplemented by preprocessing agents as well as a stateful orchestration process. Instead of considering preprocessing as one unified step or a complete exploration of the fixed space of pipelines, this paper aims to dynamically choose preprocessing modules based on properties of a specific data set.

Thus, the main research gap that this study attempts to bridge concerns the absence of systematic analysis of a specific multi-agent architecture based on an L.L.M for adaptive data preprocessing, taking into account several types of data and feature preprocessing tasks. Along with the analysis of the ability to use the resulting dataset, the efficiency, computational costs, reproducibility, and limitations of such a multi-agent architecture are also considered.

3. Proposed Methodology

A. System Overview

The suggested system will be an L.L.M-based multi-agent system which would help in automating the preprocessing of C.S.V files. The main goal of the suggested system would be to minimize human labor involved in identifying preprocessing needs of a particular dataset and generating a repeatable preprocessing pipeline.

The framework supports the upload of a C.S.V dataset using either the upload of a file or the use of a publicly available C.S.V U.R.L. This is followed by analyzing the input to obtain metadata regarding the dataset and its data quality. Depending on this metadata and the stated purpose of the machine learning model, the L.L.M-based orchestrator identifies the applicable pre-processing steps to be performed by the appropriate agents.

The entire process flow may be depicted as:

Equation 1 summary: This expression depicts a process flow that transforms an input data set into a preprocessed data set. The sequence proceeds from the input data set to a data set profile, which the orchestration module combines with a specified learning task to select multiple preprocessing agents that produce the final output.

where D stands for the input data set, P open parenthesis D close parenthesis for the data set profile, T for the learning task specified, O for the orchestration module, A i is the chosen preprocessing agent, and D prime is the obtained preprocessed data set.

The design of the system adopts a modular design where each preprocessing task is delegated to specific agents. These agents are controlled via the LangGraph technology that implements stateful execution and allows modeling the preprocessing process as a computational graph.

B. Multi-Agent Architecture

The significant elements of the suggested architectural design include:

- Dataset Profiling: It analyses the data in the input file and obtains necessary structural and statistical details which are useful in decision making later on.

- Orchestrator: employs an L.L.M to understand the dataset profile and the learning objective and ascertain what kinds of preprocessing need to be done.

- Missing-Value Agent: has a suitable approach to deal with missing values.

• Outlier Agent: Anomalies in numbers are detected and managed based on the data set properties.

- Categorical Encoding Agent: turns categorical variables into an appropriate representation for machine learning algorithms.

- Numerical Transformation Agent: uses proper transformations for numerical attributes where necessary.

- Dimensionality Reduction Agent: reduces the number of dimensions in the feature space when the properties of the dataset or learning task require it.

- Class Balancing Agent: solves class-imbalance problems in classification datasets.

- Artifact Generation: provides a processed data set along with documents and executables that describe the process of pre-processing.

This is because of the modular nature, which enables the development of the preprocessing capability in isolation, but at the same time ensures a shared execution state. Another benefit that comes with the modular approach is that it allows evaluating each agent separately.

C. Dataset Profiling

Profiling the data occurs before any decision-making regarding pre-processing. The reason for this is to create a structured view of the data for the orchestrator, rather than forcing the language model to think about the raw C.S.V file directly.

During the profiling step, features like the number of rows and columns, data type of features, statistics on missing values, distribution of numeric data, categorical features, and target variable are taken into account. Such observations form the basis of the type of preprocessing needed.

The resulting profile may be described as follows:

Equation 2 summary: This expression defines a profile represented as a set of six distinct information components. These components consist of structural, missing-value, categorical-feature, numerical-feature, potential outlier, and target variable information.

where S is structural information, M is missing-value information, C is categorical-feature information, N is numerical-feature characteristics, O is potential outlier information, and Y is information related to the target variable.

The profile is then sent to the orchestration phase. This distinction between the examination of the data set and the execution of the pre-processing enables pre-processing to be informed by facts about the data set.

D. L.L.M-Based Orchestration

The orchestration module will decide which preprocessing agents should be used. It takes as input the profile of the dataset and the learning task, either classification or regression.

Decision making may be defined as:

Equation 3 summary: This expression defines the decision making process to determine the set of preprocessing agents chosen by the orchestration engine. It computes this set by applying a large language model function to the profile of the data set and the learning task.

where P D is the profile of the data set, T is the learning task, and A star denotes the set of preprocessing agents chosen by the orchestration engine.

The purpose of the L.L.M here is not to operate on the whole data set. Rather, it serves as a decision-making module that will decide what kind of preprocessing features can be applied; the actual preprocessing is done by the programmatic preprocessing modules.

The distinction is important since it allows for the integration of L.L.M's flexible processing with deterministic computational processes. The L.L.M interprets different properties of the dataset, whereas specialized agents perform these transformations based on known data processing methods.

E. Stateful Agent Execution

The selected agents will execute using a LangGraph-based workflow. A common state is kept through execution, meaning that information created during one phase can be provided to another.

In theory, the state may be depicted as:

Equation 4 summary: This expression defines the state at a specific execution step as a set of five components. The set consists of the current dataset, the dataset profile, the learning goal, the accumulated execution information, and the intermediate result.

Here, D t denotes the current dataset at the time of the t-th execution step, P denotes the dataset profile, T denotes the learning goal, L t denotes the accumulated execution information till the tth execution step, and R t denotes the intermediate result.

The execution of the agent A i causes a change in the state by:

Equation 5 summary: This expression calculates the next state based on the current state. The process involves applying the transformation of agent i to the current state input to produce the subsequent state output.

This state is subsequently transferred to the next chosen agent, thus providing for the chaining of pre-processing activities while retaining knowledge of the transformations that have been applied.

Statefulness also contributes to ensuring consistency between the different levels of the preprocessors. The agent that operates on numeric attributes, for instance, can use the knowledge generated from profiling, while later agents operate on the transformed representation created at previous levels.

F. Missing-Value Imputation

One of the most frequent issues of quality in data in tables is missing values. The agent dealing with missing values has to recognize missing observations and apply the proper imputation technique.

The imputation method is determined by the nature of the feature that has been affected – numeric and nominal features require different approaches. This is why the agent takes into account the type of feature as well as the statistic information before performing the appropriate operation.

The transition may be described as:

Equation 6 summary: This expression calculates the missing data values. It performs this transformation by applying a chosen imputation technique to the combination of the original data and the missing value information.

where M is the missing-value information determined by profiling, and I is the imputation technique chosen.

The goal is to create a database whereby missing values have been accounted for without tampering with the valid values.

G. Outlier Handling

Outliers can have a significant impact on statistical analysis and machine learning – especially machine learning that is distribution-sensitive. The outlier handling module locates potentially problematic numeric values and treats them appropriately.

Outlier detection and treatment depend on the features of the particular data set; outliers are detected and treated by modification or elimination based on the selected technique. The processed data set is denoted as:

Equation 7 summary: This expression defines the calculation for outlier detection. It produces the outlier output by applying a chosen outlier handling method to the missing data and the set of numeric features.

In which X is the set of numeric features being examined for outliers and O is the chosen outlier handling method.

It is good to have an anomaly-specific agent that will help in performing pre-processing of anomalies without affecting other pre-processing activities.

H. Categorical Feature Encoding

Machine learning algorithms need categorical data to be presented in numerical form. Categorical encoding agent discovers categorical attributes and performs encoding transformation.

Let C be the collection of categorical attributes. The process of encoding can be written as:

Equation 8 summary: This expression calculates the encoded outliers. It performs a transformation where the outlier output and a constant are processed using a chosen encoding technique.

Where E is the chosen encoding technique.

The reason for this is that the agent acts in the context of the entire workflow rather than applying the categorical encoding universally, thus allowing the preprocessing workflow to identify whether the dataset contains categorical attributes or numerical values predominantly.

1. Numerical Transformation

Numerical features can have varied scales or distributions, which affect the behavior of machine learning algorithms used further. The task of the numerical transformation agent is to apply appropriate transformations if necessary.

A general numeric transformation is expressed as:

Equation 9 summary: This expression performs a general numeric transformation. It applies a chosen transformation function to an original numerical feature vector to produce a transformed output vector.

where X represents the original numerical feature vector and g is the chosen transformation.

This transformation is included in the workflow if the nature of the data set and the learning objective necessitate it, which is true to the overall principle of not performing needless preprocessing steps.

J. Dimensionality Reduction

The dimensionality reduction procedure can be articulated as:

Data sets with many features might require dimensionality reduction to reduce computational complexity, and in some instances, eliminate redundancies as well.

Equation 10 summary: This operation calculates a resultant low-dimensional matrix. It does this by applying a chosen dimensionality reduction method to the original feature matrix.

where X is the original feature matrix, h is the chosen dimensionality reduction method, and Z is the resultant low-dimensional matrix.

In the above mentioned workflow, dimensional reduction has been considered as a special functionality and not as a mandatory part of the workflow – its usage being determined by the orchestration itself and the properties of the input data.

K. Class Balancing

Class imbalance is one of the most frequently occurring issues in the context of supervised learning, whereby some classes have far fewer instances compared to other classes. The occurrence of a highly imbalanced dataset during training could result in poor classification for the minority classes.

The class-balancing module solves this problem by ensuring that an appropriate class-balancing technique is used when the situation demands one.

Define the target variable as Y. Then the balancing equation is given by:

Equation 11 summary: This expression calculates a balanced dataset. It uses a chosen class balancing technique to transform the original dataset and the target variable into the final result.

where B represents the chosen class balancing technique.

This module is conditionally activated in classification workflows, and it does not apply to regression scenarios.

50. Preprocessing Artifacts and Reproducibility

A critical element of the proposed system is that the preprocessing step does not only involve creation of the transformed dataset but creation of artifacts describing the steps performed in the process as well.

The output is in the form of a preprocessing report, the python code used for preprocessing, and a Jupyter notebook. This allows users to have a history of what transformations were chosen by the system and implemented.

The output of the entire system can thus be described as:

Equation 12 summary: This expression defines the final output of the system as a set of five distinct components. These components consist of the processed dataset, the preprocessing report, the generated code file, the generated notebook, and the reported data quality assessment.

where D' is the processed dataset, R is the preprocessing report, C is the generated code file, N is the generated notebook, and Q is the reported data quality assessment.

This approach makes the proposed framework distinct from a system that outputs just a transformed dataset because this way, it offers more information that can be useful for understanding the preprocessing steps.

M. End-to-End Workflow

The entire workflow starts from a moment where the user supplies a dataset in C.S.V format and defines the task for the machine learning model. At first, the data is loaded and profiled to get some insights on the data. The profiling result is sent to the orchestrator that is based on the L.L.M.

The chosen agents then run using the LangGraph pipeline. The agent is fed the state, executes its task, modifies the state, and passes it on to the next stage. This goes on till all selected pre-processing tasks are completed.

The last step is to produce the processed data set and its preprocessing artifacts. The entire process can thus be summarized as:

Math summary: This expression describes a sequential data processing pipeline. The workflow transforms an initial dataset through a profile stage, orchestrates it into an optimal plan, executes that plan to produce a revised dataset, and finally generates the final output.

This design enables an organized manner through which to integrate the L.L.M decision-making approach with programmatic data analysis. Experiments presented in subsequent sections will validate the claim that there are benefits of such a design approach compared to others.

4. Experimental Setup

This part will explain the methodology adopted for the validation of the multi-agent data preprocessing framework that was developed. The intention of this is to examine the extent to which this architecture can detect and correct different data quality problems, while at the same time measuring its computational costs.

A. Research Questions

The experiments will be based on the following research questions:

- R.Q.1: How effective is the multi-agent system proposed in recognizing and correcting common problems associated with data quality in table data?

- R.Q.2: How does the multi-agent approach stand against traditional preprocessing techniques and the use of a single agent?

- R.Q.3: Are there better results from specialized preprocessing agents as opposed to a single agent system architecture?

- R.Q.4: How much is the computational cost of the L.L.M-driven multi-agent architecture, in terms of computation time and model utilization?

- R.Q.5: How does performance of the system differ depending on the characteristics of the datasets and the nature of data quality issues added to them?

This serves as the starting point for the experiments to be discussed in the subsequent subsections.

B. Datasets

The experiment should employ several publicly available tabular datasets that have different feature types, different sizes, different class distributions, and different properties in general. It is crucial to use more than one dataset for this purpose because a certain preprocessing technique can work well with one dataset but not with another one.

These chosen datasets need to have an equal amount of numeric and categorical attributes wherever possible. The classification data sets should also have different balances of classes so that the class balancing module can be tested individually.

For each data set, the original form should be preserved for future use as a benchmark. This way, a manipulated data set will be created to see how the system is able to detect and solve particular data quality issues.

Dataset collection for experiments can be illustrated as:

Equation 14 summary: This expression defines a dataset collection for experiments. It represents the collection as a set containing multiple individual datasets used during the testing process.

where D i refers to datasets used during the testing process. For the data sets utilized in the corruption experiments, the clean data set is the truth:

Equation 15 summary: This expression describes a transformation where a clean dataset is processed to create a corrupted version. The controlled corruption process acts on the clean input dataset to produce the resulting corrupt dataset.

where C k denotes a controlled corruption process and D i,k corrupt denotes the corrupt dataset that resulted from

The above approach makes it feasible to measure the performance of a data pre-processing system using a reference benchmark instead of depending solely on subjective measures of data quality.

C. Controlled Data-Quality Corruption

In order to determine the effectiveness of the proposed system in cleaning the data, corruption needs to be introduced in certain sets of data. Corruption will change the dataset but preserve the original one for verification purposes.

The major types of corruption covered by this paper are:

- Missing Values: Randomly substituting missing values for a predetermined percentage of valid observations.

- Duplicate Records: Repetition of certain observations to create duplicates.

- Numerical Outliers: Adding extreme numeric values to the chosen numeric variables.

- Categorical Inconsistencies: Categorical variables modification to produce inconsistencies.

- Feature Distribution Problems: applying controlled modifications to numerical feature distributions when required.

- Class Imbalance: Class distribution changes within classification data sets in order to test the class balancing module.

Various levels of corruption may be employed to test for robustness. For instance, missing data may be included in low, medium, and high levels. Such levels may be formulated in respect of other forms of corruption, if necessary.

The corruption index may be expressed as:

Equation 16 summary: This expression calculates the corruption index. It is computed by dividing the number of corrupted samples by the number of eligible samples.

where N corrupted represents the number of samples that have been corrupted using some corruption process and N eligible represents the number of samples that can be corrupted.

The use of controlled corruption allows for an objective way of assessing the ability of the preprocessing system to address data quality issues.

D. Baseline Methods

The suggested multi-agent system must be evaluated relative to several baselines in order to see if the difference in performance is due to the multi-agent nature of the system or just to conventional preprocessing.

1) Raw Dataset Baseline: Baseline one uses the corrupt dataset without automated data cleaning, which sets the impact of the data quality issues on the chosen downstream task.

2) Conventional Preprocessing Baseline: Baseline two comprises a traditional programmable data preprocessing pipeline built using standard data processing libraries. The pipeline must use defined rules in performing tasks like missing values replacement, categorical variables encoding, numerical scaling, among others.

The traditional baseline is a handcrafted workflow and serves as a benchmark for a non-agentic method.

3) Single-Agent Baseline: The single agent baseline approach can be used where applicable. The idea behind this approach is that only one agent which uses L.L.M will receive information about the dataset and learning goals and will decide which preprocessing to perform.

This baseline is especially crucial as it eliminates the effect of agent decomposition. The single-agent and multi-agent models ought to be configured similarly where possible.

4) Proposed Multi-Agent System: The suggested system employs the specialized agent architecture introduced in Section 3. The L.L.M-based orchestrator chooses suitable preprocessing agents and executes their actions via the LangGraph pipeline.

The four setups can hence be described as:

Equations 17, 18, 19, 20 summary: This set of expressions defines four distinct experimental setups for comparison. These configurations include the raw dataset, conventional preprocessing, a single large language model agent, and the proposed multi-agent system.

Comparisons between these configurations permit the research to distinguish the contributions of preprocessing per say, L.L.M's' decision-making process, and multi-agent specialization.

E. Evaluation Metrics

Several categories of metrics are used in the evaluation, since the quality of preprocessing cannot be measured using one metric alone.

1) Data-Quality Metrics: In case of corruptive experiments, the result of the data processing system can be contrasted against the original uncontaminated dataset. For different types of corruptive attacks, the following measures could be applied: precision, recall, F.1-measure, and correction accuracy.

In case of error detection task, precision is given by:

Equation （21） summary: This formula calculates precision for an error detection task. It divides the number of true positives by the sum of true positives and false positives.

and recall is defined as:

Equation (22) summary: This expression calculates recall. It is computed by dividing the number of true positives by the sum of true positives and false negatives.

The F.1-score is calculated as:

Equation 23 summary: This expression calculates the F1-score. It is computed by taking two times the product of precision and recall and dividing that result by the sum of precision and recall.

Such measurements become quite handy when it comes to determining whether the system properly detects corrupt data without making any changes to correct data.

2) Data Retention and Modification: It should also evaluate how many observations are valid after the data cleaning process. This is because an excessive alteration of valid values means the cleaning process was too aggressive.

A proper data retention technique can be described as:

Equation 24 summary: This expression calculates the retention rate. It is computed by dividing the number of correctly retained observations by the total number of valid observations in the reference data set.

Where N correctly retained refers to valid observations which have not been altered wrongly and N valid is the number of valid observations in the reference data set.

3) Downstream Machine Learning Performance: Since preprocessing serves the purpose of assisting in machine learning eventually, it makes sense for the study to also consider how preprocessing affects model performance.

For classification datasets, suitable metrics include:

• Accuracy

• Precision

• Recall

• F.1-score

R.O.C-A.U.C, where applicable

For regression datasets, suitable metrics include:

• Mean Absolute Error (M.A.E)

• Mean Squared Error (M.S.E)

• Root Mean Squared Error (R.M.S.E)

times R squared

The downstream evaluation can be represented as:

: Equation 25 summary: This expression represents a downstream evaluation process. It shows a sequence where processed data and a selected machine learning model serve as inputs to determine a final performance result.

where M represents a selected machine learning model.

For the purpose of making the comparison valid, a similar downstream model and the same strategy for training and testing need to be applied for each preprocessing approach.

F. Execution Time and Computational Cost

Use of L.L.M's and multiple agents can result in increased computational overhead. Hence, preprocessing performance must be measured along with execution efficiency.

The following measurements should be recorded:

• Total preprocessing execution time.

• Number of L.L.M calls.

- Input and output token usage, when available.

• Estimated A.P.I cost, when applicable.

• Number of agents executed.

• Number of preprocessing operations performed.

The total execution time can be represented as:

Equation 26 summary: This expression calculates the total execution time. It sums the profiling time, orchestration time, the execution times of all individual agents, and the time required to generate final artifacts.

where T profile represents profiling time, T orchestrate represents orchestration time, T agent i represents the execution time of the i-th agent, and T artifact represents the time required to generate the final artifacts.

This analysis enables the advantages of the multi-agent architecture to be examined in conjunction with its computation complexity.

G. Ablation Study

Ablation tests have to be performed to evaluate the contribution of each individual component in the proposed architecture. Comparison has been made between the whole system and the systems where certain components/agents are missing.

For instance, when the overall system includes profiling, orchestration, preprocessing agents, and artifact generators or validators, then different experiments may be done to determine the effects of eliminating each component separately.

The general ablation formulation is:

Equation 27 summary: This expression calculates the difference in performance during a general ablation formulation. It subtracts the performance after removing a selected component from the performance of the complete system.

where Performance full represents the performance of the complete system and Performance minus component represents the performance after removing a selected component.

This ablation study aims to determine whether the good performance of the entire system comes about because of specialization of the agents or merely from the pre-processing procedures.

H. Experimental Repetition and Reproducibility

The goal of the ablation study is to determine if the observed system performance is due to meaningful specialization of the agents or because of the preprocessing itself.

For an experiment repeated r times, the mean performance can be calculated as:

Equation 28 summary: This expression calculates the mean performance for an experiment. It sums all the input performance values and divides that total by the number of repetitions.

and the standard deviation can be reported as:

Equation 29 summary: This expression calculates the standard deviation to report the variation of an agentic system. It computes the square root of the average squared difference between each individual input value and the overall mean, scaling the sum by the total count minus one.

Both the average and variation must be reported since just one execution being successful will not sufficiently reflect the consistency of the agentic system.

For better reproducibility, the experiment environment needs to log dataset version, corruption configuration, model name and version, configuration for that model, dependencies, random seeds used if any, and preprocessing artifacts created during the process.

1. Experimental Comparison

For the last experimental comparison, the following three parameters should be taken into account:

1) Effectiveness: Whether the framework is able to recognize and correct for the quality issues in the data and enhance machine learning outcomes.

2) Adaptability: whether the system is capable of choosing suitable pre-processing functions for datasets with varying structures and qualities.

3) Efficiency: whether the benefits derived from using the multi-agent pre-processing are worth the extra computing time and cost associated with L.L.M's.

The results from these experiments will serve in answering the research questions posed at the outset of this section. Comparisons among the different experimental settings – conventional, single-agent, and multi-agent – will serve to give information on the role of L.L.M-based decision making and agent decomposition.

5. Implementation Details

This section discusses the implementation of the proposed multi-agent preprocessing pipeline and also gives the information needed for reproducing the system. The implementation includes FastA.P.I backend, LangGraph agent pipeline, decision-making using L.L.M's via Groq, and data processing libraries in Python.

A. Software Architecture

This system architecture is implemented using a web-based service that comprises three main layers, namely the A.P.I layer, the agent orchestration layer, and the data-processing/artifact generation layer.

FastA.P.I is used in the implementation of the A.P.I layer, which receives datasets, initiates preprocessing tasks, triggers background processing, serves job results, and generates artifacts. The agent orchestration layer is based on LangGraph and handles the coordination of execution of the preprocessing agents via a shared state. The data-processing layer implements data transformations using pandas, NumPy, scikit-learn, and imbalanced-learn.

The key software components are classified into the below modules:

- agents/: contains the specialized preprocessing agents and shared utilities.

- core/: contains the shared agent state, LangGraph workflow, and L.L.M configuration.

- api/: contains the FastA.P.I application, streaming interface, and browser interface.

- reports/: contains report generation, quality scoring, and notebook generation.

- tests/: contains automated tests for the preprocessing pipeline.

This modular system separates decision making, pre-processing, A.P.I processing, and results production.

B. Technology Stack

The backend codebase runs on Python version 3.10 or greater. FastA.P.I is used to make H.T.T.P interfaces, whereas Uvicorn acts as the app server. LangGraph is used to build the stateful preprocessing pipeline, whereas LangChain offers the L.L.M integration layer.

The L.L.M implementation is done using Groq A.P.I via LangChain's Groq. The model settings that are used by default are based on the llama 3.3 70B model with a temperature setting of 0.1 and a maximum output of 1024 tokens.

Data processing procedures are mainly carried out using the pandas and NumPy libraries. Scikit-learn offers utilities for feature scaling, P.C.A, feature selection, and other statistical transformations. The smote procedure is implemented using the imbalanced-learn library.

The system makes use of Server-Sent Events (S.S.E) to provide real-time execution feedback for the browser-based U.I. The nbformat package is used to create Jupyter notebooks using Python code.

C. Shared Agent State

The agents use the shared state which is present in core/state.py. The state holds both the original dataset and the current representation of the dataset.

The main state variables are the dataset path, learning goal, original dataframe, transformed dataframe, chosen agents, active agent, report of each agent, created preprocessing results, warnings, and errors.

The shared state can be represented as:

Equation 30 summary: This expression defines the shared state as a set of specific components. It combines the original and current processed dataframes with the learning goal, the selected sequence of agents, generated reports, errors, and warnings.

where D 0 is the original dataframe, D t is the current processed dataframe, T is the learning goal, A star is the selected sequence of agents, R is the set of reports generated by agents, E is errors and W is warnings.

Each preprocessing function accepts the current state, modifies the dataframe when necessary, logs its processing details, and then passes back the modified state.

This shared state architecture makes it possible for the output of one preprocess to serve as the input of the next preprocess without any direct communication between the agents themselves.

D. LangGraph Workflow

The pre-processing process is represented using a directed graph with LangGraph. This graph comprises the following nodes:

1) Orchestrator

2) Profiling Agent

3) Imputation Agent

7) Dimensionality Reduction Agent

9) Report Generator

It is the role of the orchestrator to decide what agents must be executed. Profiling generally has to be executed first since other decisions and reports depend on dataset quality information.

Following the profiling stage, the process of routing proceeds logically through the chosen preprocessing agents. The routing skips those preprocessing agents which have not been chosen by the orchestrator.

The resulting execution sequence can therefore be represented as:

Equation 31 summary: This expression defines a linear execution sequence for a workflow. The process starts with orchestration, moves to profiling, proceeds through a series of selected preprocessing agents, and concludes with report generation.

where O represents orchestration, P represents profiling, A i j represents a selected preprocessing agent, and R represents report generation.

In case there are no other preprocessors selected, the process goes straight from orchestration to report creation, following the completion of the needed profiling step.

E. Agent Selection Mechanism

Orchestrator makes a concise summary of the input data before making the request to the L.L.M. Summary consists of dimensions of the dataset, number of missing values, numerical columns, object columns, duplicates, and numerical skewness.

The L.L.M gets this input data along with the learning goal and told to pick the right preprocessing agents for the dataset.

Let the set of available agents be:

Equation 32 summary: This expression defines a set of available agents. The set contains seven distinct elements representing profiling, imputation, outlier handling, encoding, numerical transformation, dimensionality reduction, and sampling.

where P denotes profiling, I denotes imputation, O denotes outlier handling, E denotes encoding, T denotes numerical transformation, D denotes dimensionality reduction, and S denotes sampling.

The orchestrator produces a selected subset:

Equation 33 summary: This expression defines a subset relationship between two sets of agents. It states that the selected agent names produced by the orchestrator are a subset of all acceptable agent names.

Selected agent names are extracted from the response of the L.L.M and checked against the list of acceptable agent names.

There is also a mechanism in place to handle failure cases. In the event that the response from the L.L.M does not parse into an agent list, the pipeline fails over to the available set of preprocessing agents.

F. Dataset Profiling Implementation

It is important to mention that the profile agent represents the statistical profiling done by the preprocessing workflow. It determines the number of rows and columns, column names, data type, number of missing values and percentage, number of duplicate rows, numerical summary, number of outliers, category cardinality, and top categories.

Target columns are identified through structural properties of the data set. Columns with binary values and columns that have few unique values are identified as possible classification targets. Other heuristics apply in determining the target columns for dimensional reduction and regression.

Additionally, the profiling component will produce L.L.M-powered preprocessing insights based on the statistics calculated. The insights will be included in the profiling report and will become available in the generated outputs.

G. Missing-Value Processing

The imputer agent checks whether there are any missing values. In case the current dataframe lacks any missing values, the agent logs the fact that the operation was not performed.

Upon identifying the presence of missing values, L.L.M offers various methods that include mean imputation, median imputation, mode imputation, constant value imputation, column deletion, and row deletion.

For numeric attributes, mean and median techniques make use of properties of the column in question statistically. For mode imputation, the most common value is used. The constant imputation is done using the value Unknown.

The agent documents the selected strategy, affected columns, pre and post dataframes sizes, and remaining missing values.

H. Outlier Processing

Outlier agent identifies outliers in numeric columns through the use of interquartile range and z score statistics. Detection based on interquartile range is performed to determine if outlier processing is needed.

The supported methods include winsorization, log transformation, deletion of rows, and retention of detected data points.

Values lying beyond the bounds computed from I.Q.R are truncated in the case of winsorization, whereas in the case of log transformation, the shift value is added when required so that the log value could be calculated from the non-positive observations.

The agent creates the distributions of before and after for specific affected numeric attributes.

1. Categorical Encoding

If there are still no object-type columns, then the task of the encoding agent will not be performed.

Other four strategies for transformation are used along with keeping the original encoding: one hot encoding, label encoding, frequency encoding, and dropping columns.

The one-hot encoding technique is applied through the use of pandas dummy variable creation. The label encoding converts the unique and sorted categories into integer numbers. The frequency encoding converts the categories into the frequency of their occurrence in the column.

Encoding Agent: The encoding agent stores the selected operation and distribution details of selected categorical attributes.

J. Numerical Transformation

The transformation technique works on numerical columns after the initial preprocessing steps have been applied. The available transformation methods are standard scaler, min-max scaler, robust scaler, log transform, and keeping the original values.

Standard scaling is performed through StandardScaler, while min-max scaling and robust scaling are done using MinMaxScaler and RobustScaler respectively. Log transformation employs a shifted natural log function

The transformation agent is thus able to modify feature representations numerically depending on the preprocessing option chosen by the orchestration process.

K. Dimensionality Reduction

Dimensionality reduction is incorporated as an optional phase. Dimensionality reduction will be employed if there are ten or more columns in the data set. The incorporation includes P.C.A (Principal Component Analysis) and Select K Best.

The target variable is dropped before the transformation in P.C.A. The obtained components are denoted by the names of the features like P.C.1, P.C.2 and the rest.

For SelectKBest, the implementation uses the anova F-test through f classif for classification tasks and f regression for regression-oriented objectives.

It is run conditionally since the dimensionality reduction process is not applicable to all kinds of tabular data.

50. Class-Balancing Implementation

The sampling agent is designed for classification tasks and is skipped when the learning objective indicates regression.

Frequencies of classes, percentages of classes, and ratio between majority and minority classes are computed by the agent. Ratio higher than 1.5 is considered as the presence of class imbalance.

The suggested techniques are as follows: random oversampling, random undersampling, smote, and keeping the class distribution unchanged.

Oversampling and undersampling employ a fixed random state of 42. smote is employed through the imbalanced-learn package. When the use of smote fails, then the alternative approach used is oversampling.

M. L.L.M Reliability and Retry Mechanism

As there is a possibility of failure in the connection with the L.L.M because of temporary faults in the network, rate limit, and availability issues, the program provides a retry function.

L.L.M wrapper will retry for rate-limit error, timeout error, connection error, and H.T.T.P 502, 503, and 504 response code. Rate-limit errors will have increasing wait time between attempts, while other errors will be attempted at a different backoff period.

Using the retry logic makes it possible to handle temporary external errors without stopping the entire preprocessing pipeline.

N. Reproducible Artifact Generation

Once the preprocessing step is finished, several artifacts get generated by the system from the created workflow. These artifacts include the preprocessed C.S.V file, the Markdown preprocessing report, the Python preprocessing code, and the Jupyter notebook.

The automatically created report captures information on the preprocessing process and its impact. The Python code is the implementation of the preprocessing process and the notebook is an interactive implementation that can be used to examine and analyze the data.

This artifact-generation stage is represented as:

Equation 34 summary: This expression defines the artifact generation stage as a transformation of the processed dataset. The output consists of a set containing the processed dataset, a generated Markdown report, a generated Python script, and a generated Jupyter notebook.

where D prime C.S.V is the processed dataset, R M.D is the generated Markdown report, C P.Y is the generated Python script, and N i.py.nb is the generated Jupyter notebook.

O. Quality Scoring

The system also calculates a quality-heuristic measure meant to capture the differences between qualities of selected dataset attributes prior to and following preprocessing.

The score is produced at the end of the preprocessing stage and it is part of the completed job response.

Since this measure is more heuristic in nature than a universally accepted measure of data quality, it must be understood to be an internal measure and not an external standard. In light of that, this experiment must take into account established measures like precision, recall, F.1-score, and machine learning as key evaluation metrics.

P. Asynchronous A.P.I and Progress Monitoring

The operation of pre-processing will be performed in the background such that an I.D of the process can be provided to the A.P.I without making the client wait until the whole process is completed.

Each job is assigned an identifier which is a string of eight characters. A job output folder specific to the job is also assigned to each job.

During execution, agents generate progress events using Server-Sent Events. Such events are agent-start, agent-complete, complete, and error events. These events are then picked by the browser and used to update the workflow status.

The process will enable users to view the development of the multi-agent process as the pre-processing process takes place.

Q. Error Handling

The pipeline keeps individual error and warning fields within the shared state. Hence, non-critical problems can be registered without having to stop the whole process flow.

The A.P.I makes the distinction between the ongoing jobs, the jobs which have been processed, and the jobs whose results cannot be accessed. The artifacts generated are placed in separate directories depending on the job to avoid confusion between different preprocessing jobs.

This design will provide some level of fault isolation without losing any information about diagnostic information of executed tasks.

R. Implementation Summary

This approach includes a combination of L.L.M-based orchestration and deterministic programmatic transformations. The large language model is mostly employed for reasoning about dataset properties and determining proper transformation techniques, while the special agents perform these transformations using existing Python libraries.

The resulting architecture can be summarized as:

Math summary: This expression describes a sequential flow through an architecture. It transforms a dataset into a profile, which then moves through large language model orchestration to result in specialized agent thirty five.

Such separation of decision-making and data processing is one of the main features of the system under consideration. It ensures flexibility of the reasoning module while maintaining explicitness of preprocessing procedures. “latex

6. Results

In this part, we describe the experimental evaluation of the suggested framework for data preprocessing using an L.L.M-enabled multi-agent approach. It is compared to traditional data preprocessing and a setup based on a single agent under various data quality levels. The assessment is centered on preprocessing efficiency, machine learning results, overhead, and contribution of each component.

A. Overall Data Cleaning Performance

The first experiment examines how well each of the preprocessing techniques performs in identifying and fixing artificial data quality issues. The corrupted data sets are compared against the clean data sets in order to establish whether the preprocessing technique fixes the artificial data quality issues.

Table 1 summarizes the overall cleaning performance.

The findings will be reviewed to establish whether the proposed architectural approach offers higher accuracy in the detection and correction of data quality issues than the benchmark approaches. Both successes and failures will be considered in analyzing the findings.

Table I summary: Overall data cleaning performance across three methods: Conventional Preprocessing, Single-Agent System, and the Proposed Multi-Agent System. The table contains no numerical values for Precision, Recall, or F1-score for any of these methods.

B. End-to-End Application Case Study

Apart from the benchmark tests, an end-to-end test is done using the deployed application itself. A real dataset is uploaded through the application interface, and the entire multi-agent process takes place. The objective of doing this test is to see how the entire system behaves starting from dataset upload till the generation of the final output dataset.

Profiling of the input data is done at the initial stage using the input data to ascertain the structure and statistical properties of the data. The L.L.M orchestrator uses the profiling data and the given learning objective to select the appropriate preprocessors for the data. The chosen preprocessors will then work on the shared dataframe state.

Table 2 summarizes the changes observed between the input and processed datasets.

Interpretation of the changes in size of the dataset depends on the kind of preprocessing methods that are applied by the system. For instance, an increase in the number of observations does not imply that there has been recovery of data in case the Validation is due to the Repositioner at Repositionalistic Artifacts, like oversampling or smote.

Likewise, a decrease in the number of observations could be a consequence of explicit deletion or under-sampling. Changes in the number of columns can come about through categorical encoding or dimensionality reduction.

The present case study is thus an illustration of whether the system generates a processed data set, as well as the way that the chosen operations for data preprocessing impact the nature of the file.

Table II summary: An end-to-end application case study comparing data properties before and after processing. The table includes categories for number of rows, number of columns, missing values, duplicate rows, detected outliers, minority and majority class samples, and a quality score, though no specific values are provided for these properties.

C. Performance Across Data-Quality Problems

This is because the framework consists of specific agents for handling different types of preprocessing problems. This means that the framework evaluates the performance of each problem in isolation as well as combinations of multiple problems.

Table 3 presents the comparison across individual data-quality problems.

This comparison is employed to establish whether the specialization of pre-processing agents is always advantageous for different categories of data quality issues or whether the performance of the architecture is dependent on certain characteristics of the database.

Table III summary: Performance metrics across different data-quality problems, including Missing Values, Outliers, Categorical Errors, Class Imbalance, and Mixed Problems, are listed for Conventional, Single-Agent, and Multi-Agent approaches. However, no numerical data is provided for any of these problems across the three categories.

D. Downstream Machine Learning Performance

The efficiency of preprocessing is also tested using the subsequent application of machine learning techniques. The same set of data is subjected to different preprocessing techniques. After applying the preprocessing technique, the same data is used to learn the same machine learning model.

In classification datasets, the metrics considered include accuracy, precision, recall, F.1-score, and R.O.C-A.U.C. In regression datasets, the metrics considered include M.A.E, R.M.S.E, and R squared .

Table 4 provides the structure for the downstream evaluation.

This evaluation helps to look at the problem from another angle since a preprocessing pipeline should eventually generate data that can be used for further machine learning. processing. But the performance of the model is considered along with the data quality measures since higher predictive performance doesn't necessarily mean correct recovery of initial data.

Table IV summary: Downstream machine learning performance for Dataset 1 and Dataset 2 across four methods: Raw, Conventional, Single-Agent, and Multi-Agent. The table contains no recorded values for Accuracy, F1-score, or ROC-AUC for any of the datasets or methods.

E. Agent Selection Accuracy

Another assessment looks at whether the orchestrator makes the right choice of preprocessing methods based on the nature of the dataset.

The chosen agents are compared to those which are regarded as necessary given the corruption environment and dataset properties for every particular experiment.

Accuracy of agent selection may be written as:

Equation 36 summary: This expression calculates the accuracy of agent selection. It is computed by dividing the number of correctly selected preprocessing components by the total number of required components for the experiment.

where N correct represents the number of required preprocessing components correctly selected by the orchestrator and N required represents the number of components required for the corresponding experiment.

Table 5 provides the structure for reporting these results.

The experiment tests directly the decision-making part of the proposed architecture without being affected by the efficiency of the preprocessing algorithms used.

Table V summary: Agent selection performance across Dataset 1, Dataset 2, and Dataset 3. No data is provided for the number of required agents, selected agents, or selection accuracy for any of the listed datasets.

F. Computational Performance

The multi-agent design necessitates L.L.M inference and perhaps multiple runs of the agents. Hence, computation performance is analyzed through total time of execution, number of L.L.M queries, tokens used, and inference cost estimation when available.

The computational analysis is performed to identify if any enhancements in pre-processing performance come with a reasonable increase in the cost of execution.

Table VI summary: Computational performance for Conventional Preprocessing, a Single-Agent System, and a Multi-Agent System shows no recorded data for runtime, LLM calls, tokens, or cost across any of the three methods.

G. Ablation Study

The purpose of an ablation study is to test the impact of each architectural element. The entire architecture is contrasted against a variety of architectures in which certain elements have been eliminated.

The difference between the complete system and each ablated configuration can shed light on the role of the particular component. A large drop in performance for each ablated configuration could mean that the component has a role in the overall performance of the system.

Table VII summary: An ablation study comparing the Full Multi-Agent System against configurations without Orchestration, without Profiling, without Specialized Agents, and without Validation. However, no numeric values for Precision, Recall, or F1-score are provided for any of these configurations.

H. Robustness to Increasing Corruption

The robustness of the proposed system is tested with respect to the severity of the corruption in the data introduced. Different amounts of data missing values or outliers are introduced, but the original non-corrupted data set remains the baseline.

The correlation between the corruption level and the preprocessing efficiency is stated as:

Equation 37 summary: This expression calculates the preprocessing performance as a function of the corruption ratio. It defines a direct relationship where the corruption ratio serves as the input to determine the resulting performance value.

where P denotes preprocessing performance and rho denotes the corruption ratio.

From the findings, it will be clear whether the suggested architecture is still viable even with increasing data quality issues.

Table VIII summary: Performance metrics under Low, Medium, and High corruption levels for Conventional, Single-Agent, and Multi-Agent configurations. However, all data cells for these categories are empty, indicated by dashes.

1. Failure Analysis

Quantitative measures alone cannot completely justify the behavior of an L.L.M-based pre-processing system. The representative examples of failure are discussed in order to identify the instances when the orchestrator makes a wrong choice of agent, an agent makes a wrong choice of transformation, and some pre-processing step is omitted.

The failure analysis will classify observed failures based on their source, which includes:

• Incorrect agent selection.

- Incorrect preprocessing strategy selection.

- Unnecessary transformation of valid data.

• Failure to detect a data-quality problem.

- Execution failure within a preprocessing agent.

- L.L.M response parsing or orchestration failure.

Example cases will be considered along with the features of their datasets and systems.

J. Summary of Results

The entire experimental study will be used to respond to the research questions posed in Section 4. Specifically, the analysis will show whether the suggested multi-agent approach increases the efficiency of the preprocessing process, whether the use of dedicated agents offers any advantage compared to a single agent, how the system performs under various data quality levels, and whether the extra processing cost is worthwhile.

The final results will be presented using aggregate statistics and statistics per data set to avoid making any conclusion from one experimental condition. ^{“}

7. Results

Experimental testing will focus on the assessment of the efficiency of the proposed multi-agent-based preprocessing scheme under various quality conditions and properties of datasets. The results are structured with respect to the research questions formulated in Section 4. Specifically, the evaluation focuses on the comparison of the proposed multi-agent solution with traditional approaches and single-agent solutions, as well as on overheads and contribution of the components.

A. Overall Preprocessing Performance

The first experiment assesses the capability of the suggested solution in terms of detecting and solving various types of data-quality issues. For the case of corruption-related experiments, the performance of each method must be compared against its clean counterpart.

Table 9 presents the overall cleaning performance across the evaluated datasets.

It is important to discuss the findings based on the average results as well as the variations in different data sets. In addition, the cases where multi-agents' approach reveals data quality issues that cannot be addressed by baseline approaches are especially important. Cases when multi-agent approach underperforms should also be discussed for a comprehensive analysis.

: Table IX summary: Data cleaning performance across three datasets, comparing Conventional, Single Agent, and Multi-Agent methods. However, the table contains no numerical data for Precision, Recall, or F1-score for any of the methods across Dataset 1, Dataset 2, and Dataset 3.

B. Performance by Data-Quality Problem

The second analysis looks at the performance in different categories of data quality issues. This analysis is very vital since the system will have specialized agents for various forms of pre-processing.

Problem Performance Comparison Table 10 serves as the basis for comparison.

The aim of this analysis is to test whether specialization delivers a consistently positive impact on pre-processing, or whether the benefits of the multi-agent system are limited to specific problem classes.

Table X summary: Performance by data-quality problem across Conventional, Single Agent, and Multi-Agent problem types. For every listed data-quality problem, including Missing Values, Outliers, Categorical Errors, Class Imbalance, and Mixed Problems, no performance data is provided for any of the three problem types.

C. Downstream Machine Learning Performance

The quality of any preprocessing pipeline has to be assessed ultimately based on how it impacts downstream machine learning algorithms. Hence, any preprocessing setting needs to be evaluated with respect to the same machine learning algorithms and train-test split procedure.

For classification tasks, the primary comparison should include accuracy, precision, recall, and F.1-score. For regression tasks, appropriate metrics such as M.A.E, R.M.S.E, and R squared should be reported.

Table 11 provides the structure for this comparison.

An improvement in the downstream model performance would serve as an indication that the transformation of the input data into features results in better representations to feed into the machine learning model. Nevertheless, it would be misleading to view the downstream performance as an indicator of cleaning accuracy since a transformation may enhance predictions without correcting the data.

: Table XI summary: Downstream machine learning performance across Dataset 1 and Dataset 2. For both datasets, accuracy, F1-score, and AUC are evaluated using four different methods: Raw, Conventional, Single Agent, and Multi-Agent; however, no numerical values are provided for any of these metrics.

D. Computational Overhead

The employment of L.L.M-based orchestration and various special agents creates some extra computational burden. For that reason, execution time and the number of models employed have to be considered.

Table 12 provides a framework for reporting these measurements.

Results should be viewed as a compromise between the efficiency of the preprocessing phase and its computational cost. A multi-agent approach will need more steps of inference than an ordinary pipeline, and this should be taken into account when assessing the practical feasibility of the approach.

: Table XII summary: A comparison of computational and LLM usage across Conventional, Single Agent, and Multi-Agent methods. The table contains no recorded values for runtime, LLM calls, tokens, or cost for any of these three methods.

E. Ablation Results

Ablation study is performed to measure the impact of individual features in the model architecture that is developed. Every test involves disabling one feature while maintaining everything else identical as far as possible.

Table 13 provides the structure for reporting the ablation results.

Table XIII summary: An ablation study of the proposed architecture comparing the Full System against four variants: Without Profiling, Without Specialized Agents, Without Orchestration, and Without Validation. However, the table contains no reported values for Precision, Recall, or F1-score across any of these configurations.

F. Robustness Across Corruption Levels

In order to study the robustness property, tests need to be done by varying the level of corruptions. For instance, some values can be missed intentionally at various proportions without changing the original clean data set.

The resulting performance can be represented as:

Equation 38 summary: This expression calculates performance as a function of corruption level. The result is determined by applying a transformation to the input corruption level value.

where rho represents the corruption level.

A good preprocessing process must retain its efficiency despite an increase in the magnitude of data quality issues. However, it is important to document the reduction in performance in case of extreme corruption of data since such a level of corruption might render recovery impossible.

G. Failure Analysis

Besides overall evaluation measures, there should be qualitative analysis conducted on instances when the proposed method makes wrong or improper preprocessing decisions. Such failure cases can involve improper recognition of problems with the quality of data, unnecessary transformation operations, inappropriate dealing with ambivalent categories, improper outliers handling, or improper features transformation.

Such examples provide an insight into the restrictions of L.L.M-based decision-making process. The agent may appropriately recognize a need to do some preprocessing but use a wrong transformation for a particular dataset. In addition, orchestration decision might lead to activation of a needless agent, which will consequently produce a useless change to the data.

As such, the failure analysis will need to provide representative samples, along with an explanation of why they failed. The results of this analysis complement those of the quantitative data and help pinpoint areas for improvement.

8. Discussion

The framework suggested will tackle the problem of automated data pre-processing not as a linear series of processes, but as an ensemble of decision-making processes. The aim of the experiments is to check whether this approach can offer some advantage over traditional approaches.

A. Effectiveness of Agent Specialization

One major assumption underlying the architecture described above is that agents with particularized responsibilities perform better in preprocessing duties than a single component that performs all preprocessing tasks. Each agent is constructed based on a certain category of preprocessing tasks such that the decision-making process involved in implementing them becomes relatively easy.

In case the outcome of the experiment shows better results from the multi-agent system, it will indicate the role of task decomposition in effective preprocessing. On the other hand, in case the results of both systems turn out to be similar, then there should be some advantage in using multiple agents to justify their increased complexity.

B. Adaptability to Dataset Characteristics

The need for the suggested approach is to account for the heterogeneity of real-life data sets. Each set might require different preprocessing procedures and using all of them might result in unnecessary changes.

The mechanism introduced will solve this problem through selecting agents based on the properties of the datasets and the defined learning goal. As such, the experimental analysis should consider the ability of the system to activate the appropriate pre-processing components in relation to the various properties of the datasets.

An adaptation process which proved to be successful would imply that L.L.M-based decision making can act as a rather flexible layer in addition to the preprocessing processes. In case of wrong agent selection, the need for further validation techniques or limitations in decision making should be expected.

C. Trade-off Between Effectiveness and Efficiency

Multiple L.L.M components result in a tradeoff between flexibility and efficiency. Traditional preprocessing pipelines can perform deterministic actions with low overhead, while an agentic approach may need more steps such as model inference.

Therefore, better performance in the preprocessing phase is not sufficient evidence to claim that the proposed system is superior in terms of practical application. Factors such as runtime, token usage, A.P.I cost, and the execution of agents also need to be taken into account.

The usefulness of the suggested approach will hence depend on the particular use case scenario. In analytical scenarios where significant manual preprocessing is required in terms of work, extra computational overhead may be tolerable. In scenarios involving large-scale production pipelines with latency or cost constraints, an ordinary deterministic pipeline may be more appropriate.

D. Reproducibility and Transparency

The ability to create the preprocessing code, notebook, and report adds another layer to assess the suggested framework. Instead of simply generating the outputted dataset, the framework also documents the process through which the data was preprocessed.

This is especially significant when it comes to data preprocessing since users might have to be aware of the transformation that was applied to the original dataset in order to use the generated data in their machine learning model. But merely the creation of artifacts does not assure the reproducibility of results. This is due to certain aspects that are dependent on model version, prompts, dependencies, random seeds, and other factors such as the behavior of external A.P.I's.

E. Implications for Automated Data Preparation

Such an architectural structure shows how a pattern of dividing reasoning based on an L.L.M from deterministic data processing tasks can be achieved. Language modeling mainly focuses on analyzing dataset properties and choosing proper preprocessing techniques, whereas other entities carry out these tasks.

This segregation can perhaps form an effective basis to expand the capability of automated data preparation systems. More agents can be added in tasks like feature selection, feature engineering, data validation, schema matching, or even specialized transformations without affecting the orchestration infrastructure.

Increasing the number of agents will also result in more architectural complexity. Hence, for future work, there should be an analysis to determine if having a higher number of specialized agents consistently results in more effective performance. Increasing the number of agents will also result in more architectural complexity. Hence, for future work, there should be an analysis to determine if having a higher number of specialized agents consistently results in more effective performance.

9. Limitations

There are numerous drawbacks to the proposed method which must be taken into account while analyzing the outcomes.

First of all, the quality of preprocessing decisions is determined in part by the capabilities of the base language model. Language models might come up with different decisions even for the same dataset, depending on their version or inference setting.

Secondly, the existing system focuses mainly on tabular data available in C.S.V format. Hence, the framework and evaluation does not prove its usefulness in dealing with unstructured data, time series data, graph data, and others.

Thirdly, automated pre-processing decisions could be influenced by ambiguous or inadequate dataset information. Dataset profiling provides statistical and structural information; however, it does not necessarily provide domain-related information for identifying whether a certain value is wrong or not.

Fourth, the use of more than one agent can result in increased computation compared to the deterministic processing pipeline. This increase in computation could make the system unsuitable for latency-sensitive production systems.

Moreover, controlled corruption experiments can help identify the ground truth but may fail to cover all aspects of the data quality problem that occur in real-world data sets because the issues with real data may be related and may depend on the data generation process.

Ultimately, the evaluation of a preprocessing algorithm cannot depend only on its performance in the subsequent process of machine learning. In some cases, an algorithm may increase the prediction accuracy while incorrectly transforming valid examples. This is why all aspects listed above should be evaluated jointly.

10. Conclusion

This paper introduces an L.L.M-based multi-agent system designed for automated preprocessing of tabular data. The architecture consists of decomposition of the preprocessing task into individual agents combined with an L.L.M-based orchestration process to decide on the applicable steps based on data properties and the learning goal. It involves a stateful LangGraph pipeline of various preprocessing tasks that outputs not only preprocessed data but also artifacts to describe the whole preprocessing process.

The experimental setting will be structured in such a way that the proposed framework will be tested from a variety of angles – data quality enhancement, M.L algorithms performance, flexibility, efficiency, and individual contributions. Comparing the proposed framework to traditional methods and single agent configurations can help establish whether specialization and coordination yield any gains.

The findings from these experiments will reveal how far the suggested architecture can help to enhance automated data preparation at an acceptable computational expense and replicability level. No matter whether the multi-agent framework is consistently superior to the chosen benchmarks, the study may shed light on advantages and disadvantages of leveraging agentic architectures with L.L.M's for data preparation tasks.

Potential avenues for future research could involve the inclusion of further data quality detection features, improved validation methods, preprocessing agents for specific domains, better coordination among the agents, and extensive testing on larger datasets. Potential areas for further investigation could include adaptive model selection, human-in-the-loop validation, and cost-effective model orchestration for production environment use.

References

[1] C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown, “Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms,” in Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 847–855, 2013.

[2] M. Feurer, A. Klein, K. Eggensperger, J. T. Springerberg, M. Blum, and F. Hutter, “Efficient and robust automated machine learning,” in Advances in Neural Information Processing Systems, vol. 28, 2015.

[3] R. S. Olson and J. H. Moore, "TPOT: A tree-based pipeline optimization tool for automating machine learning," in Proceedings of the Workshop on Automatic Machine Learning, pp. 66–74, 2016.

[4] M. Feurer, K. Eggensperger, S. Falkner, M. Lindauer, and F. Hutter, “Auto-sklearn 2.0: Hands-free AutoML via meta-learning,” Journal of Machine Learning Research, vol. 23, no. 29, pp. 1–61, 2022.

[5] T. B. Brown et al., “Language models are few-shot learners,” in Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020.

[6] S. Yao et al., “ReAct: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations, 2023.

[7] Q. Wu et al., “AutoGen: Enabling next-gen LLM applications via multi-agent conversation,” arXiv preprint arXiv:2308.08155, 2023.

[8] S. Hong et al., “MetaGPT: Meta programming for multi-agent collaborative framework,” in International Conference on Learning Representations, 2024.