Konstantin Kosev, Head of RDI at Bright MR, shares a practical perspective on how large language models are being applied in survey workflows and why human judgment remains essential for reliable results.
Language Models as a Driver of Automation
Language models are capable of highly effective analysis of any text. They are significantly faster than humans at extracting logical relationships and identifying inconsistencies or gaps in structure and reasoning.
Many organizations possess large volumes of internal knowledge - training materials, historical projects, or established solutions. When this information is provided to large language models as a reference context, it allows them to generate content and make decisions within a specific domain.
These capabilities can be used both for initial text analysis and gap detection, and for building structured workflows that support iterative, automated decision-making until a specific task is successfully completed.
Opportunities and Challenges in Market Research
Artificial intelligence is both a source of concern and a major opportunity for the market research industry.
As models improve in domain understanding, context retention, and cost efficiency, new possibilities emerge for their application in the creation and management of online surveys. This is turning a traditionally stable operational area into a rapidly evolving one, after nearly a decade of relatively slow technological change.
Limitations of Classical Analysis Approaches
Over the past decade, there have been attempts to replace traditional data collection with preference analysis from social platforms and automated sentiment analysis.
In practice, this approach is not universally applicable.
When evaluating a new product or service, there is often no reference data or existing conversation that can support reliable modeling. In highly specialized domains, where public data is limited, researchers still rely on structured research methods and direct data collection. In such cases, traditional surveys and interviews remain essential.
Economic Pressure and the Need for Automation
The operational challenge in recent years is well known within the industry: the cost of project management, survey scripting, data processing, and respondent panels.
At the same time, rapid progress in AI has created a perception among clients that automation should make research faster and less expensive. This expectation increasingly translates into pressure for shorter timelines, faster scripting, and continuous price reduction.
Technological Barriers in Survey Scripting
Automating the scripting process has always been extremely challenging.
The main difficulty lies in the nature of the input. Regardless of the format in which a questionnaire is delivered, it ultimately comes as unstructured text. Each client uses different conventions, structures, and ways of describing logic.
The absence of a unified structure makes it difficult for systems to reliably identify where questions begin and end, how routing is defined, and how filters or instructions should be interpreted.
Some companies have addressed this by requiring clients to follow strict templates. While this can improve machine recognition, it introduces operational friction. Clients must adapt their working processes, learn new formats, and maintain different standards when working with multiple vendors.
In practice, clients are rarely willing to change established habits simply to make processing easier for a supplier - and this is entirely understandable.
The Role of LLMs in Data Structuring
Large language models offer a practical way to address this challenge.
Although they do not inherently understand the technical specifics of survey platforms, they are highly effective at identifying logically distinct units within text. Using entity recognition and structured output methods, unorganized questionnaires can be transformed into structured data, with metadata attached to each element.
This allows unstructured input to be converted into a format that can be interpreted and processed programmatically.
The Problem with Syntax and Specific Platforms
The second major challenge is translating the extracted elements into the specific markup required by a given scripting platform. In other words, how to move from an intermediate structured format (for example, JSON with identified entities) to a ready-to-import working script.
The difficulty here is not whether the model can perform a transformation, but that it lacks platform-specific knowledge and syntax.
A language model understands what a question, an answer option, or a programming instruction represents. However, it does not inherently know how these concepts should be expressed in the final code required by a specific platform.
While modern models can generate code in many widely used programming languages with high accuracy, this does not mean they understand all proprietary or specialized formats. Language models are trained primarily on public data. As a result, generating Python or JavaScript may be straightforward, but producing the correct XML structure for a proprietary survey platform is often unreliable simply because the model has never encountered it.
Providing this platform-specific knowledge therefore becomes the second critical task.
Solutions via Specialized Methods (RAG and Fine-Tuning)
The key to making language models operational lies in narrowing their scope and enriching them with domain-specific knowledge.
Approaches such as Retrieval-Augmented Generation (RAG), prompt engineering, specialized agents, and orchestration frameworks allow the model to operate within a controlled knowledge environment. By giving the system access to curated examples and documentation, it can retrieve relevant information and refine its output based on the standards required.
This makes it possible to teach the model the nuances of survey scripting - how specific question types should be structured, how logic and filters should be implemented, and which tags or syntax must be used.
By organizing the process into structured decision flows, the system can simulate the steps a programmer would take when building a questionnaire. At each stage, the model can reference the knowledge base, resolve uncertainties, and generate output that aligns with platform requirements.
Using these methods, a large portion of standard questionnaires can already be converted into working scripts with a high degree of automation.
Where Human Expertise Remains Essential
Despite these advances, human expertise continues to play a critical role.
Even when models generate high-quality output at significantly greater speed, there are situations where no suitable example exists, instructions are ambiguous, or multiple interpretations are possible.
In these cases, human specialists provide judgment, context, and validation. They intervene when the output appears inconsistent, incomplete, or logically unclear.
For the foreseeable future, automation and expertise will operate together. Language models accelerate execution, but human oversight ensures accuracy, consistency, and alignment with the research intent.
Ultimately, human expertise remains at the center of reliable market research operations.
Konstantin Kosev, Head of RDI at Bright MR
Get in touch with us for more information: