Strengthening Trust

Automation technologies should be implemented in a socially responsible and context-sensitive manner. This means that decisions on the degree of automation must take into account both technical factors (e.g., risk of reproducing of content that infringes rights, scalability) and sociocultural aspects, such as societal power relations, the protection of minorities, political contexts, and the risk of instrumentalisation by the state.

In particular, it is important to

• Clearly define the degree of automation (fully automated, semiautomated, or only reviewed by humans) with reference to the above-mentioned factors
• Design systems so that they operate as independently as possible from specific user profiles. Automated decisions should only consider sensitive user characteristics to the extent that is minimally necessary in order to minimise bias
• Analyse cultural and social conditions in a differentiated manner (e.g., handling of LGBTIQA+ content in repressive contexts, linguistic characteristics, historically sensitive issues)
• Minimise risks to marginalised groups in automated decisions in a targeted manner.
• Consider both false positives (overblocking of legitimate content) and false negatives (failure to recognise harmful content)
• Understand social acceptance not as a uniform variable but as a pluralistic negotiation process involving the affected groups

In order to minimise incorrect decisions and avoid inappropriate or excessive automation, tasks should only be delegated to automated systems if they are technically mature, transparently verifiable, and societally responsible – especially with regard to critical, ethically sensitive, or context-dependent decisions. The following applies: The higher the degree of delegation to automated systems, the greater the responsibility to safeguard them with robust control and fallback and feedback mechanisms. A high degree of automation always means a high degree of delegation of decision-making responsibility – this correlation must be reflected on and limited, especially in the case of content that is potentially relevant under criminal law, such as threats of violence, announcements of the intention to commit mass shootings/rampage attacks, or incitement to hatred and agitation. In such cases, automatic intervention alone is neither appropriate nor responsible – additional human evaluation remains mandatory.

Technical maturity refers to the state in which a system:

• Has been proven in independent audits to make consistently accurate decisions (e.g., measured by very low false positive/negative rates, fairness metrics)
• Has been security tested to the latest standards (e.g., through adversarial testing)
• Can be continuously monitored and improved
• Is equipped with clear fallback/escalation mechanisms for special cases
• And has been validated against a training set of curated, human-reviewed decisions – ideally with representative, diversely annotated content to minimise bias

Critical decision-making processes are those that

• Pose a risk to fundamental rights (e.g., freedom of expression, data protection, discrimination)
• Could have irreversible consequences for individuals or groups
• Take place in societally highly standardised or conflict-ridden contexts
• Or concern criminally relevant content where incorrect decisions can have serious real-world consequences (e.g., public safety, prevention of violence or hate speech)

In safety-critical, particularly complex, or fundamental-rights-threatening situations, human control must be strengthened. Automated systems should not continue autonomously if there are indicators of significant risks to democratic processes, public safety, fundamental rights, or user protection. Automation must be interruptible at any time – by clearly defined intervention procedures, responsible persons, and transparent documentation. The goal is risk-adequate, ethically acceptable human-machine interaction that relies on preventive and reactive emergency mechanisms.

Timely intervention is particularly necessary when

• Safety-critical situations arise (e.g., threats to users’ physical or digital safety)
• Particularly complex situations arise in which the technical model logic conflicts with real-world contextualisation (e.g., through ambiguous language, cultural connotations, novel/unprecedented phenomena)
• Systemic risks as defined by the Digital Services Act or the AI Act arise (e.g., threats to democratic processes, targeted disinformation, discrimination relevant to fundamental rights)
• Significant impacts on individuals or groups are to be expected—especially for vulnerable or marginalised user groups

Ensure that automated moderation processes are interrupted when content exhibits a high degree of cultural, ethical, or legal complexity. In such cases, human intervention (dynamic escalation for human review) must be mandatory in order to adequately consider fundamental rights, cultural contexts, and ambiguous interpretations.

Complexity indicators are characteristics that indicate that content cannot be evaluated by automated systems. These include:

• Ambiguity of linguistic expressions (irony, sarcasm, context dependency, regional idioms)
• Cultural and religious symbolism that can be interpreted differently depending on the region or group,
• Topics that strongly affect marginalised groups (e.g., queer identities, racialised perspectives, colonial or antisemitic language elements)
• Overlaps with sensitive political contexts (e.g., elections, protest, dissent, authoritarian narratives/propaganda, war).

The list of complexity indicators shall be public, dynamic, and regularly updated with input from interdisciplinary expert groups (especially in computer science, law, and social sciences). Users, civil society organisations, and moderation teams are explicitly encouraged to suggest new indicators. A defined review process shall ensure that new suggestions are evaluated and documented in a participatory manner.

Note: These indicators shall trigger a risk-based, tiered escalation process that combines automated preliminary analyses with human review.

All systems and interfaces used in the context of content governance shall be developed with a human-centered approach. The design of systems, their techno-physical interfaces, and digital user interfaces should minimise physical and psychological stress and enable natural forms of interaction for moderators – especially in demanding, highly repetitive, or potentially stressful and disturbing contexts. Mental health is not an individual responsibility but part of the employer's duty to provide a safe working environment and care for their employees.

Who are moderators (broadly defined)?

• Internal teams & outsourced service providers
• So-called high-level expert groups
• System administrators
• Community members, if applicable, when they take on moderation functions (e.g., via platform reporting systems)
• Trusted flaggers (within the meaning of Art. 22 DSA)

What does "natural interaction" mean?

• Transparent & understandable
• Barrier-free
• Psychologically relieving (e.g., options to take a break, preview blockers for distressing content)
• Intuitive

Automated and semiautomated content moderation requires a careful balance between the protection of personal data and the consideration of contextual information necessary for fair, transparent, and nondiscriminatory decisions. All data processing steps – including the analysis of post content, metadata, usage contexts, and, where applicable, personal account information – shall be guided by the principles of data minimisation, purpose limitation, and contextual appropriateness.

In this context, contextual appropriateness means weighing the fundamental rights at stake (such as freedom of expression, data protection, protection against discrimination, or protection against violence) in a proportionate manner, taking into account the social, communicative, and technical context of a post. The collection and evaluation of so-called context data, such as visibility settings, audience targeting, communication space (public, semi-public, private), time sequence, interaction patterns, or platform architecture, may only take place if this is essential for the assessment of content.

Key principles

• Data protection in accordance with the GDPR (Art. 1,5 GDPR: Protection of natural persons with regard to the processing of personal data and on the free movement of such data)
• Principle of proportionality and balancing of rights under the DSA: Moderation decisions must be proportionate to the potential infringement of fundamental rights (Art. 14 DSA).
• Consideration of users' privacy settings (e.g., private stories, closed groups, protected profiles vs. public content) and prioritisation according to reach

(Semi)automated systems, especially in the area of content moderation and recommendation, should be designed and regulated in such a way that they detect structural exclusion, algorithmic biases, and unintended amplification mechanisms at an early stage and effectively limit them. Fairness is understood here as equal access, nondiscrimination based on human rights, and the enabling of equal participation.

Content moderation systems (CMS) have a particular responsibility in this regard: They must not only detect and remove illegal content but also ensure that their mode of operation does not indirectly disadvantage marginalised groups, for example, through higher error rates in the detection of dialects or nondominant forms of language or through the unequal removal of legal content. These systems must be transparent, accountable, and nondiscriminatory in accordance with the requirements of the Digital Services Act (DSA). Platforms are also required to provide clear rules, transparent processes, and effective complaint mechanisms.

Recommender systems (RS) (algorithmic recommendation systems for sorting, prioritising, or controlling the visibility of content) play a central role in content governance. They largely determine what content users see and what they do not. The underlying reinforcement logic is usually based on engagement rates such as likes, shares, or watch time. However, these metrics can have discriminatory side effects if, for example, they disadvantage content that is less emotional or comes from groups whose contributions receive less feedback – such as people with disabilities, FLINTA/LGBTQIA+, BIPoC, or nondominant language communities.

Content should therefore not be prioritised solely on the basis of how polarising or emotional it is. The goal is to design recommendation algorithms that do not push problematic dynamics such as toxic discourse, hate speech, or disinformation as strongly and do not structurally disadvantage diverse, contextualised content. This means designing recommendation algorithms in such a way that they enable democratic participation, diversity of opinion, and fair access in an unequal digital space.

In this context, contextualised equal treatment does not mean treating all content or users identically but rather systematically considering social inequalities, structural discrimination, and existing access restrictions. Fair algorithmic weighting requires an adaptive system design that reveals distortions and reinforcement loops, makes discriminatory effects verifiable, and enables participatory corrections.

Strengthening the professional competence, ethical confidence, and psychological resilience of moderators who work with automated systems and their effects. The focus is not only on qualification, but also on care, protection, and structural relief.

• Content moderation is high-stress work and requires professional training, pay commensurate with qualifications and workload, psychological support, relief, and supervision.
• Competence building does not equal transfer of responsibility: The responsibility for fair, functional systems does not lie with individual employees but with the organisation as a whole.

Ensuring that automated decisions are continuously reviewed and improved through human perspectives – both through internal feedback from moderators and through formalised appeal options for users. This double feedback loop should help to ensure fairness, system learning, and trust.

Decisions made by automated systems must be understandable to users, civil society organisations, academia, public authorities, and regulatory bodies. This includes the disclosure of relevant system information as well as the possibility of access, review of legality, and appeal.

In alignment with the protection of intellectual property and trade secrets, the following elements in particular should be disclosed:

• The core logic of algorithmic decision-making processes: e.g., filter criteria, scoring systems, reinforcement mechanisms, model training.
• System characteristics: rule-based systems, type of machine learning (supervised, unsupervised, reinforcement), deep learning, hash-matching, hybrid model architectures, model purpose.
• Training data used: Making them publicly available in accordance with Art. 53(1)(d) AIA; this applies in particular to the origin of the data, data categories, and possible biases.
• Within the scope of the DSA: description of automated and human decision-making steps, including decision-making bases (terms and conditions, guidelines, legal requirements), see Art. 17 DSA.
• False decisions and review practices: proportion of automated moderation, withdrawal rate, appeal procedures, systematic biases (bias detection), specific effects on data subjects.

This disclosure shall be made in accordance with the requirements of the Digital Services Act (DSA), in particular:

• Easily comprehensible information on algorithmic decision-making processes for users (Art. 15 DSA)
• Clear and specific obligations to provide reasons for moderation decisions (Art. 17 DSA).
• Risk assessments and their publication in the transparency report; existing risk-based assessments (e.g., within the framework of the AIA, internal risk analyses, or external audits) should be integrated in a meaningful way and communicated openly (Artt. 34, 42 DSA).
• Information to be provided for automated decisions (Artt. 13-15, 22 GDPR).

Individual case explanations

Structured, understandable, and accessible explanations shall be provided for all relevant individual case decisions, regardless of whether the decision was automated, made by a human, or arrived at through a hybrid process. This applies to:

• Users whose content has been removed, flagged, or deprioritised
• Users whose reports have not led to action
• Changes to a previously made decision

The explanations must be clear and specific and comply with the requirements of Art. 17 DSA, Artt. 13-15 GDPR, and Art. 86 AIA.
Decisions that have a collective impact on entire groups or subject areas should be publicly documented in aggregate form and analysed regularly (e.g., regarding the visibility of queer content or political activism).