14-Ethics.Rmd

# Ethical Issues in Assessment {#ethics}

Ethics are relevant to all of our domains as psychologists, including (but not limited to) research, teaching, assessment, intervention, and supervision.\index{ethics}

## Belmont Report {#belmontReport}

The Belmont Report informs professional ethics.\index{ethics}\index{Belmont report}
The principles from the Belmont Report include: respect for persons, beneficence, and justice.\index{ethics}\index{Belmont report}\index{respect for persons}\index{beneficence}\index{justice}

### Respect for Persons {#respectForPersons}

A key component of respect for persons is informed consent.\index{ethics}\index{Belmont report}\index{respect for persons}
It also involves special considerations to protect the welfare of vulnerable people, including children, prisoners, pregnant women, people with intellectual disability, and people who are economically or educationally disadvantaged.\index{ethics}\index{Belmont report}\index{respect for persons}
It is important to get full informed consent from clients or participants before taking actions that will affect them.\index{ethics}\index{Belmont report}\index{respect for persons}

### Beneficence {#beneficence}

The principle of beneficence is about maximizing benefit and minimizing harm, similar to the aphorism, "First, do no harm".\index{ethics}\index{Belmont report}\index{beneficence}
Clients and participants have the prerogative to make choices about whether to receive a procedure, so they need to know the benefits and harm of each procedure.\index{ethics}\index{Belmont report}\index{beneficence}
Just having the intent to help is not enough—you also need to be using a helpful technique and to have the knowledge of how to implement the technique.\index{ethics}\index{Belmont report}\index{beneficence}

### Justice {#justice}

Justice goes beyond the interaction with the individual client or participant; it deals with broader, societal concerns.\index{ethics}\index{Belmont report}\index{justice}
For instance, justice involves ensuring that the people taking the risk will also benefit.\index{ethics}\index{Belmont report}\index{justice}
For example, if a study to test the effectiveness of a medication enrolls primarily homeless people, the participant population may be taking the risk of trying the drug but not ultimately benefiting from the knowledge gained.\index{ethics}\index{Belmont report}\index{justice}

## My Ethical Advice {#myEthicalAdvice}

My ethical advice is summarized in the following two pieces of advice:\index{ethics}

1. Don't do stupid things\index{ethics}
2. Don't do bad things\index{ethics}

### Don't Do Stupid Things {#dontDoStupidThings}

First, don't do stupid things.
With respect to assessement, for instance, do not use weak assessment instruments.
The assessment techniques you use should be reliable and valid for the tasks and population of interest.
They should be based on scientific research, not by professional judgment.
Reliability and validity have many facets and all facets must be used in evaluating assessment techniques.\index{reliability}\index{validity}
Reliability and validity are characteristics of *uses* of tests, not the tests themselves.\index{reliability}\index{validity}
So, identify what is important, for example test–retest reliability versus internal consistency.\index{reliability!test–retest}\index{reliability!internal consistency}
Reliability and validity may differ for populations that vary in characteristics (e.g., age, gender, education, ethnicity).\index{reliability}\index{validity}

### Don't Do Bad Things {#dontDoBadThings}

Second, don't do bad things.
For instance, do not administer services without first getting the person's consent.\index{informed consent}
Informed consent for an assessment includes multiple components.\index{informed consent}
Ensure that the person being assessed is informed about important aspects of the testing situation.\index{informed consent}
Say what services will be administered (i.e., the nature of testing), why the testing will be administered (i.e., the purpose of testing), what the evidence is for the assessment devices, the costs and potential side effects of the assessment and the tests, the pros and cons of underdoing the assessment, and other options available.\index{informed consent}
Describe the type of information that the client will receive.\index{informed consent}
Describe any third-party involvement (e.g., insurance companies, schools, employers, court) in accessing the assessment information.\index{informed consent}
Describe the limits of confidentiality, which typically would include when someone is in danger, as defined by unreported child abuse or neglect, elder abuse, or imminent suicidality or homicidality.\index{informed consent}\index{confidentiality}
If the assessment is court-ordered or the results of the assessment are subpoenaed by a court of law, the limits of confidentiality change.\index{informed consent}\index{confidentiality}
It is also important to describe the psychologist's role relative to the client.\index{informed consent}
Be transparent about what services are experimental; do not pretend to know more than you know.\index{informed consent}
Do not make false or misleading claims or promises.\index{informed consent}

Honor the agreement implied by the explanation provided in the informed consent.\index{informed consent}
For an example of a document that I use with clients that describes the pros and cons of an intervention technique (parent management training) and other options available, see here:\index{informed consent} https://osf.io/qmnu6.

It is also important to maintain the integrity of assessment techniques.
Do not teach people how to raise their scores.
There has to be integrity to begin assessment.

### Consult, Consult, Consult {#consult}

If you are unsure how to handle an ethical dilemma, CONSULT, CONSULT, CONSULT!
In notes and reports, provide documentation of the rationale for decisions made.\index{reporting}
Professional ethics boards want to know that you are acting in good faith of the client's best interests.\index{ethics}

## APA Ethics Code {#APAethicsCode}

The American Psychological Association (APA) publishes ethics guidelines in its document entitled, *Ethical Principles of Psychologists and Code of Conduct* (hereafter referred to as the "APA Ethics Code") [@Association2017].\index{ethics!APA Ethics Code}
The latest version of the APA Ethics Code is available here: https://www.apa.org/ethics/code (archived at https://perma.cc/6F4Z-WQ57).\index{ethics!APA Ethics Code}
The current version of the APA Ethics code was published in 2017 and is available here: https://www.apa.org/ethics/code/ethics-code-2017.pdf (archived at: https://perma.cc/PF6F-QZ5C).\index{ethics!APA Ethics Code}
The APA also published "Guidelines for Psychological Assessment and Evaluation", available here: https://www.apa.org/about/policy/guidelines-psychological-assessment-evaluation.pdf (archived at https://perma.cc/FRB2-ND58).\index{ethics!APA Ethics Code}

Here, I emphasize a few of the guidelines from the APA Ethics Code with respect to assessment.\index{ethics!APA Ethics Code}
However, you should read and follow the entire APA Ethics Code.\index{ethics!APA Ethics Code}
That said, the APA Ethics Code is not up to date for clinical science.\index{ethics!APA Ethics Code}
But there are many important components.\index{ethics!APA Ethics Code}
Case illustrations of ethical guidelines from the APA Ethics Code are provided by @Campbell2010b.\index{ethics!APA Ethics Code}
Ethical issues in psychological assessment are also discussed by @Bersoff2012 and @Nagy2011.\index{ethics}

### Competence {#competence-APA}

Section 2 of the APA Ethics Code notes that you should only act withing the boundaries of your competence.\index{ethics!APA Ethics Code}
It is also important to be aware of and report conflicts of interest.\index{ethics!APA Ethics Code}

### Bases for Assessments {#basesForAssessments-APA}

Section 9 of the APA Ethics Code describes ethics guidelines for assessment.\index{ethics!APA Ethics Code}
Section 9.01 describes ethics guidelines regarding the bases for assessments.\index{ethics!APA Ethics Code}
Use techniques that are sufficient to substantiate the findings, based on strong psychometrics.\index{ethics!APA Ethics Code}
In a professional context, sit down face to face and evaluate the individual, as opposed to just relying on automated reports.\index{ethics!APA Ethics Code}
If using computerized assessments, you should have the expertise to consider the appropriateness of the interpretations.\index{ethics!APA Ethics Code}
Work to reduce or eliminate bias.\index{ethics!APA Ethics Code}\index{bias}
If using record review, explain the sources of information used for conclusions and recommendations.\index{ethics!APA Ethics Code}
Maintain a high standard when recording assessment information.\index{ethics!APA Ethics Code}
Psychologists have responsibility for maintenance and retention of their records.\index{ethics!APA Ethics Code}
Records should describe the nature, delivery, progress, and results of services, and related fees.\index{ethics!APA Ethics Code}
Take reasonable steps to maintain confidentiality of records.\index{ethics!APA Ethics Code}
When a client might want to know how records will be maintained, disclose record-keeping procedures in the informed consent.\index{ethics!APA Ethics Code}\index{confidentiality}
The duration that full records should be kept depends on whether they are adults or children.\index{ethics!APA Ethics Code}
For adults, keep full records for 7 years after the last date of service.\index{ethics!APA Ethics Code}
For children, keep full records for 7 years after the last date of service, or for 3 years after they become adults, whichever is later.\index{ethics!APA Ethics Code}
The latest version of the APA record keeping guidelines is available here: https://www.apa.org/practice/guidelines/record-keeping (archived at https://perma.cc/S37L-5VNX).
The current version of the APA Ethics code was published in 2007 and is available here: https://www.apa.org/pubs/journals/features/record-keeping.pdf (archived at: https://perma.cc/7A8U-M4TQ).

### Use of Assessments {#useOfAssessments-APA}

Section 9.02 describes ethics guidelines regarding the use of assessments.\index{ethics!APA Ethics Code}
It notes to use assessments "in light of research or evidence".\index{ethics!APA Ethics Code}
But how would research and evidence not go together?!\index{ethics!APA Ethics Code}
This phrase from the APA Ethics Code suggests that some forms of non-research-based evidence have equal footing with research in terms of deciding what assessments to use.\index{ethics!APA Ethics Code}
I would argue that assessments should be selected based on the best available scientific evidence.\index{ethics!APA Ethics Code}
And if scientific evidence does not yet exist to use an assessment device for a particular purpose with a particular population, the assessment device should be studied and evaluated before it is used to make important decisions.\index{ethics!APA Ethics Code}

The assessment should be reliable and valid for the population **and** purpose.\index{ethics!APA Ethics Code}\index{reliability}\index{validity}
When the reliability and validity has not been established, describe the strengths and weaknesses.\index{ethics!APA Ethics Code}\index{reliability}\index{validity}
Use language and competence level that is appropriate for the participant/client.\index{ethics!APA Ethics Code}
Consider the potential social consequences of testing.\index{ethics!APA Ethics Code}
High stakes decisions probably should not be made on the basis of the results of a single test.\index{ethics!APA Ethics Code}

### Informed Consent in Assessments {#informedConsent-APA}

Section 9.03 describes ethics guidelines regarding informed consent in assessments.\index{ethics!APA Ethics Code}\index{informed consent}
Informed consent should be obtained unless:\index{ethics!APA Ethics Code}\index{informed consent}

- the assessment is government mandated (which changes things a bit),\index{ethics!APA Ethics Code}\index{informed consent}
- it is implied because it is considered voluntary (e.g., when applying for a job), or\index{ethics!APA Ethics Code}\index{informed consent}
- evaluating decisional capacity and the person is not capable of providing consent; in these cases, you should still inform them of what testing will include and the purpose of assessment.\index{ethics!APA Ethics Code}\index{informed consent}

### Release of Test Data {#releaseOfTestData-APA}

Section 9.04 describes ethics guidelines regarding release of test data.\index{ethics!APA Ethics Code}
"Test data" include raw and scaled scores, a client's responses to questions or stimuli, and the psychologist's notes about the participant.\index{ethics!APA Ethics Code}
You are expected to release test data if the client provides a signed release, unless it would cause substantial harm or would be a misrepresentation of the data or test.\index{ethics!APA Ethics Code}
You may also have to release test data if there is a law or court order, but you may be able to fight these.\index{ethics!APA Ethics Code}

### Other Guidelines {#otherGuidelines-APA}

Section 9.07 notes that you should not promote the use of assessment techniques by unqualified people.\index{ethics!APA Ethics Code}
Section 9.08 notes that you should not use obsolete tests and outdated results.\index{ethics!APA Ethics Code}

### Maintain Test Security (Integrity) {#maintainTestSecurity-APA}

Section 9.11 describes ethics guidelines regarding maintaining test security (integrity).\index{ethics!APA Ethics Code}
Maintaining test integrity involves protecting test instruments, protocols, questions or stimuli, and manuals from unauthorized access.\index{ethics!APA Ethics Code}
If releasing test data, take care to release only the scores of copyrighted tests; do not release the copyrighted tests themselves.\index{ethics!APA Ethics Code}
For example, if performing intellectual testing, try to release the subscale scores and summary scores (e.g., *T* scores) without releasing test stimuli.
Also, do not coach people on how to perform better on questions, and do not let them take tests home.\index{ethics!APA Ethics Code}

## AERA Guidelines {#AERAguidelines}

## Clinical Report Writing {#clinicalReportWriting}

Write every report as if it will be used in a court of law.\index{report writing}
You want it to be able to stand up in court.\index{report writing}
Always identify the source of each inference (e.g., "Per the client's report, ...").\index{report writing}
Do not write a self-reported statement as if it were a factual statement.\index{report writing}
Instead of writing, "The client cut herself", write "The client reported that she cut herself".\index{report writing}
Do not overstate findings.\index{report writing}
Information from reports can be used to make decisions that significantly impact people's lives, so it is important not to misrepresent information.\index{report writing}
Recognize factors, such as language, culture, and the situation, that could have impacted the results, and note them in the report.\index{report writing}\index{culture}
Recognize limitations in the recommendations.\index{report writing}

## Open Science {#openScience}

There has been a replication crisis in psychology and social science, in which the findings from many studies have failed to replicate when independent researchers attempt to replicate the studies' findings [@OSC2015].\index{open science}\index{replication}\index{replication!failure to replicate}
Due to the replication crisis, there has been a movement toward open, transparent science where researchers pre-register their methods, and share their data and analysis scripts.\index{open science}\index{replication}\index{replication!failure to replicate}
Open practices reduce threats to replicability and increase the public accessibility of science.\index{open science}\index{replication}
The movement to open practices in science is known as the open science movement.\index{open science}

The open science movement seeks to increase the replicability and accessibility of science through three primary methods.\index{open science}\index{replication}
First, the open science movement seeks to improve transparency, i.e., what researchers are actually doing in their studies and analyses.\index{open science}
Second, especially when researchers have confirmatory hypotheses, the open science movement seeks to restrict researcher degrees of freedom so that the researchers do what they said they would do rather than contingent analysis decisions to get a desired result.\index{open science}
Similar to a garden of forking paths (see Figure \@ref(fig:gardenForkingPaths)), there are so many decision points in a study, and different decisions can lead to drastically different results.\index{open science}
Third, the open science movement seeks to prevent ethically questionable research practices.\index{open science}
Ethically questionable research practices may include [@John2012]:\index{open science}\index{ethics!questionable research practices}\index{ethics}

- not reporting all dependent variables\index{ethics!questionable research practices}
- deciding to collect more data after looking at the data to see whether the results were significant\index{ethics!questionable research practices}
- not reporting all of a study's conditions\index{ethics!questionable research practices}
- stopping collecting data earlier than planned because the expected result is detected\index{ethics!questionable research practices}
- rounding down a *p*-value\index{ethics!questionable research practices}
- selectively reporting studies that worked\index{ethics!questionable research practices}
- deciding whether to exclude data after looking at the impact of doing so on the results\index{ethics!questionable research practices}
- reporting an unexpected finding as having been predicted from the start (known as HARK-ing: **h**ypothesizing **a**fter the **r**esults are **k**nown)\index{ethics!questionable research practices}
- claiming that results are unaffected by demographic variables when one is actually unsure\index{ethics!questionable research practices}
- falsifying data\index{ethics!questionable research practices}
- *p*-Hacking: multiple testing, selective reporting, and misreporting of the true effect size\index{ethics!questionable research practices}

To advance reproducibility and replicability in science, researchers are encouraged to provide greater transparency and sharing of their research.\index{open science}\index{replication}
For instance, one way to provide greater transparency and to limit the researcher's degrees of freedom is to (pre-)register a study.\index{open science}\index{pre-registration}
There is a continuum of study registration that includes pre-registration, co-registration, and post-registration, depending on when the registration occurs relative to data collection and analysis [@Benning2019].\index{open science}\index{pre-registration}
Study pre-registration involves publicly posting a study's design, hypotheses, methods, materials, and data analysis plan prior to data collection.
Co-registration occurs after data collection starts but before data analysis begins.\index{open science}\index{pre-registration}
Post-registration occurs after data analysis has begun.\index{open science}\index{pre-registration}

Although pre-registration is most commonly used with confirmatory research in which you have a directional hypothesis, you can also pre-register exploratory research in which you have no directional hypothesis.\index{open science}\index{pre-registration}
Pre-registration helps to make it more explicit what tests are confirmatory (based on a priori directional hypotheses) and which are exploratory.\index{open science}\index{pre-registration}
It is important to interpret the results of confirmatory and exploratory tests differently, because you are more likely to get false positive results if conducting lots of exploratory tests.\index{open science}\index{pre-registration}
You can also pre-register decision trees that allow flexibility for multiple approaches depending on the data.\index{open science}\index{pre-registration}

Some journals see the value of pre-registered studies and publish registered reports.\index{open science}\index{pre-registration}
A registered report is a study that is evaluated based on the pre-registered methods, hypotheses, and data analysis plan, and if the pre-registered study is accepted, the eventual paper will be accepted (as long as the researchers followed the pre-registered plan) regardless of whether the findings supported the hypotheses.\index{open science}\index{pre-registration}
Registered reports of exploratory analyses are sometimes called exploratory reports.\index{open science}\index{pre-registration}

There are many tools that aim to advance the open science movement.\index{open science}
A key tool is the Open Science Framework (OSF; https://osf.io).\index{open science}\index{Open Science Framework}
The OSF is a free, open source platform that supports the transparency of scientific research.\index{open science}\index{Open Science Framework}
There are numerous ways that researchers can make use of the OSF to advance the aims of open science.\index{open science}\index{Open Science Framework}
Here is a template for a pre-registration of a study on the OSF:\index{open science}\index{Open Science Framework} https://osf.io/2hrfy.
Researchers can (pre-)register their study on the OSF by posting their hypotheses, study methods, and data analysis plan.\index{open science}\index{Open Science Framework}\index{pre-registration}
In addition to the OSF, other website for pre-registration include AsPredicted (https://aspredicted.org), PROSPERO (for systematic reviews; https://www.crd.york.ac.uk/prospero), and ClinicalTrials.gov (https://clinicaltrials.gov).\index{open science}\index{pre-registration}
In addition, researchers can use the OSF to post manuals, protocols, lab notebooks, data, data processing syntax, statistical analysis code, detailed results, computational notebooks, and paper preprints [@Tackett2019a].\index{open science}\index{Open Science Framework}
A computational notebook (e.g., Jupyter notebook, R Markdown, Quarto) is a document in which the analysis code, output, and text are weaved together, so the reader can see how each statistical result was obtained for reproducibility.\index{open science}
An example of an R Markdown computational notebook is on the book's page on the OSF: https://osf.io/nkyve (the associated `.Rmd` file that generated the html is here:\index{open science}\index{Open Science Framework} https://osf.io/fxrzm).
An example of a Quarto computational notebook is also on the book's page on the OSF: https://osf.io/8ybtk (the associated `.qmd` file that generated the html is here:\index{open science}\index{Open Science Framework} https://osf.io/zbruf).
In addition to the OSF, other website for sharing materials include Databrary (which is particularly useful for sharing videos; https://nyu.databrary.org), Zenodo (https://zenodo.org), the Dataverse Project (https://dataverse.org), and figshare (https://figshare.com).\index{open science}

After pre-registration, you can share a preprint of your paper on a preprint server like [arXiv](https://arxiv.org), [PsyArXiv](https://psyarxiv.com), [bioRxiv](https://www.biorxiv.org), [SocArXiv](http://socarxiv.org), [EdArXiv](https://osf.io/preprints/edarxiv), [MetaArXiv](https://osf.io/preprints/metaarxiv), or [medRxiv](https://www.medrxiv.org).\index{open science}\index{pre-registration}\index{preprint}
For an example of a preprint, see here:\index{open science}\index{preprint} https://psyarxiv.com/gtsvw.
Posting a preprint disseminates the paper without waiting for peer review and the publishing process.\index{open science}\index{preprint}
It also provides free access to papers that might otherwise be hidden behind journals' paywalls.\index{open science}\index{preprint}
Also, it helps establishes who was first to perform work on a topic.\index{open science}\index{preprint}

There are now journals dedicated to publishing replication studies, including *ReScienceX* (http://rescience.org/x) and to publishing null findings, including the *Journal of Articles in Support of the Null Hypothesis* (https://www.jasnh.com).\index{open science}\index{replication}

Nevertheless, open science does not provide a one-size-fits-all solution to the vast diversity of scientific methods.\index{open science}
For instance, there may be important ways needed to adapt open science and pre-registration practices to longitudinal research [@Petersen2024].\index{open science}\index{pre-registration}

## Conclusion {#conclusion-ethics}

According to the [Belmont Report](#belmontReport), professional ethics includes [respect for persons](#respectForPersons), [beneficence](#beneficence), and [justice](#justice).\index{ethics}\index{Belmont report}\index{respect for persons}\index{beneficence}\index{justice}
It is also important to follow the APA Ethics Code [@Association2017] and AERA guidelines (@AERA2014).\index{ethics!APA Ethics Code}\index{Standards for Educational and Psychological Testing}
It is also important to produce a replicable science, which requires better science, including use of measures with strong psychometrics ([reliability](#reliability) and [validity](#validity)), in addition to not engaging in questionable research practices.\index{replication}\index{validity}\index{reliability}\index{ethics!questionable research practices}
The [open science](#openScience) movement, including pre-registrations, sharing data, preprints, and research materials may help improve the replicability of science.\index{open science}\index{pre-registration}\index{preprint}

## Suggested Readings {#readings-ethics}

@Association2017; @AERA2014; @Bersoff2012; @Nagy2011; @Campbell2010b