{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"toc": true
},
"source": [
"Table of Contents
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Fundamentals of data and coding for psychology"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Applying your new knowledge to some scenarios"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this final class in this series of lessons on data and coding for psychology, you will be applying some of the knowledge and skills that you have learned in previous lessons to some scenarios in which they might be of use in psychology research."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Learning outcomes covered so far"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, let's look at an inventory of all the learning outcomes that we have addressed thus far.\n",
"This is a reminder of all the things that you have learned, and also gives a quick overview if you need to go back to a lesson to review some of the details."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Getting started with notebooks\n",
"\n",
"* Create, run, and change the type of notebook cells.\n",
"* Switch between 'Command mode' and 'Edit mode'.\n",
"* Describe the differences between `Markdown` and `Code` cells.\n",
"* Use `Markdown` cells to produce formatted prose.\n",
"* Use `Code` cells to run Python code.\n",
"\n",
"#### Data associated with psychology research\n",
"\n",
"* Identify the data associated with a research study by reading the journal article reporting its results.\n",
"* Describe the types of data that are associated with conducting research in psychology across a range of specialities.\n",
"* Use a notebook to store and share written work.\n",
"\n",
"#### Digital representation\n",
"\n",
"* Describe how data is represented in memory on computers.\n",
"* Use functions and methods in Python.\n",
"* Interpret binary sequences in different ways.\n",
"\n",
"#### Data storage and navigation\n",
"\n",
"* Describe the components of filenames and file paths.\n",
"* Navigate a filesystem using absolute and relative paths.\n",
"* Develop a data storage/organisation strategy for a research project.\n",
"\n",
"#### Representing integer numbers, with an application in images\n",
"\n",
"* Use `numpy` to create, manipulate, and access one- and multi-dimensional arrays of integers.\n",
"* Plot array-based data using methods appropriate to their dimensionality (line, scatter, image plots).\n",
"* Apply `for` loops to allow statements to be repeatedly executed.\n",
"\n",
"#### Representing decimal numbers, with an application in sounds\n",
"\n",
"* Compute with non-integers using the `float` datatype.\n",
"* Generate and interrogate sounds using `numpy` arrays of floats.\n",
"* Compute statistical tests and use simulations to demonstrate their properties."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Scenarios"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will now go through a series of semi-realistic psychology research scenarios, and you will have the task addressing some of the requirements for conducting the research—often, but not always, using Python.\n",
"\n",
"A few points:\n",
"\n",
"* I say 'semi-realistic' because I have tried to cobble together aspects from a few different areas of psychology, most of which I only have a vague familiarity with. They certainly shouldn't be taken as examples of meaningful studies or of good research practices.\n",
"* Each task includes a 'Potential solution' section, and many also have a 'Hints' section; I suggest only using these as necessary.\n",
"* You will often not know the answer immediately from memory—you will typically need to review the relevant lesson and your notes.\n",
"* The scenarios are independent, so you can skip a task that you are having trouble with (and come back to it later).\n",
"* There are also quite a few, so feel free to concentrate on the ones you find most interesting.\n",
"* Some aspects are marked as 'Advanced'; those in particular are good candidates for skipping on the first pass through."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Code review"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You are working on a project with a fellow researcher, and they have written some code to generate and save an image of random noise.\n",
"However, it is not working and they have asked you to take a look at their code.\n",
"\n",
"Your task is to inspect their code and make it run as required (without actually saving anything)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pathlib\n",
"import numpy as np\n",
"\n",
"# I want to generate an image that is 100 pixels wide and 50 pixels high\n",
"my image = np.random.randomint(low=0, high=1, size=(100px, 50px))\n",
"\n",
"# I want to save it in a directory called \"images\" above the current directory\n",
"# with the filename \"image.txt\"\n",
"path = pathlib.Path(\"c:\\project\\images\\\\image\" .txt)"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"my image = np.random.randomint(low=0, high=1, size=(100px, 50px))"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"Problems:\n",
"\n",
"* `my image`: can't have spaces in variable names.\n",
"* `np.random.randomint`: the function is called `np.random.randint`.\n",
"* `high=1`: want the random integers to be up to 255.\n",
"* `size=(100px, 50px)`: Python doesn't know about `px`.\n",
"* `size=(100px, 50px)`: image sizes are (height, width) and not (width, height)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"my_image = np.random.randint(low=0, high=256, size=(50, 100))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"path = pathlib.Path(\"c:\\project\\images\\\\image\" .txt)"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"Problems:\n",
"* `\"c:\\project\\images\\\\image\"`: use of backslashes without attention to the interpretation as control characters\n",
"* `\"c:\\project\\images\\\\image\"`: use of absolute paths despite requirement being better suited to relative.\n",
"* `\"c:\\project\\images\\\\image\" .txt`: malformed syntax"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"path = pathlib.Path(\"../images/image.txt\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### EEG markers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You are planning on running an EEG study, in which a continuous record of electrical signals are recorded from the scalp of your participant.\n",
"You are presenting your trials on a different computer to the one that is recording the EEG, and you want a way of being able to indicate the precise onset time of a trial within the EEG record.\n",
"\n",
"The EEG system provides a cable that you can connect to your computer for this purpose.\n",
"It has 8 'pins', and the state of each pin ('low'/0 or 'high'/1) can be set by sending an integer to the cable using Python.\n",
"The first pin is used to indicate that you want a marker to be set in the file; this only happens when the first pin goes from 'low' to 'high'.\n",
"The remaining pins are used to set the value of the marker (the 'marker code') as an unsigned integer.\n",
"\n",
"For example, the moment that the signal on the first pin goes from low to high, the system reads the values of the other seven pins and uses those values to store a number at that point in the EEG record.\n",
"\n",
"In developing your study, you realise that you can understand the operation of the cable using the binary representations of integers that you have covered in these lessons.\n",
"That is, each pin is a different 'bit', with the first pin being the first bit and the other pins being the remaining seven bits (going from left to right).\n",
"\n",
"You also realise that you need to be able to answer the following:\n",
"\n",
"* What does the following binary representation mean with respect to the state of the pins? `00101111`.\n",
"* How many unique marker codes are available to use?\n",
"* What integer number do you need to send to register a marker code of 76, assuming the first pin is 'low' before you send the number?\n",
"* Remembering that a marker is only recorded when the first pin goes from low to high, you realise that you need to 'reset' the first pin each time you record a marker code. What number do you need to send to reset the first pin?"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solutions"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* What does the following binary representation mean, in terms of the state of the pins? `00101111`.\n",
"\n",
"The first two pins are 'low', the third is 'high', the fourth is 'low', and the fifth through eighth pins are 'high'."
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* How many unique marker codes are available to use?\n",
"\n",
"The first pin is used to signal that a marker code is to be stored, so there are seven pins available to set the marker code.\n",
"Each pin can be either 0 or 1, so there are $2^7 = 128$ potential marker codes."
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* What number do you need to send to register a marker code of 76, assuming the first pin is 'low' before you send the number?\n",
"\n",
"We want to end up with a marker of 76.\n",
"In binary, this is:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"bin(76)"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"But, we also want to set the first bit, so we actually want to send the integer corresponding to `11001100`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"int(\"11001100\", base=2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* Remembering that a marker is only recorded when the first pin goes from low to high, you realise that you need to 'reset' the first pin each time you record a marker code. What number do you need to send to reset the first pin?\n",
"\n",
"The first pin is reset whenever the first binary digit is `0`.\n",
"That happens whenever the number is less than 128, so any number that is between 0 and 127 will work."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Face database"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For an upcoming experiment, you would like to show images of faces.\n",
"You decide that you want to use the [AR Face Database](http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html), so you obtain permission to use it and then go ahead and download the files.\n",
"\n",
"However, you soon run into a problem.\n",
"The database is a series of files with names like `w-060-9.raw`, and no image viewer on your computer is able to display them.\n",
"\n",
"Having a closer read of the website, you see that it says:\n",
"\n",
"> All images are stored in 10 different CD-ROMs as RGB RAW files (pixel information). Images are of 768 by 576 pixels and of 24 bits of depth."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Your task is to interpret this information to answer the following questions, which will help you work out how to use the files:\n",
"\n",
"* What is '24 bits of depth' likely to mean?\n",
"* Assuming 1024 kilobytes in a byte, how many kilobytes would each image contain?\n",
"* What would the shape of each image be if represented using a multidimensional array?\n",
"* (Advanced) Is there enough information in these two sentences to allow you to view the images?"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Hint"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* You may want to review the 'Digital representation' and the 'Representing integer numbers, with an application in images' lessons."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* What is '24 bits of depth' likely to mean?\n",
"\n",
"Images typically have three channels (red, green, and blue) that are each represented by 8-bit unsigned integers.\n",
"Thus, '24 bits of depth' is likely to refer to 3 colour channels times 8 bits per channel.\n",
"\n",
"* Assuming 1024 kilobytes in a byte, how many kilobytes would each image contain?\n",
"\n",
"Each pixel is 24 bits, which is 3 bytes (24 / 3).\n",
"There are 442,576 pixels (768 x 576), meaning there are 1,327,104 bytes (442576 x 3).\n",
"That means there are 1,296 kilobytes per image (1327104 / 1024).\n",
"\n",
"* What would the shape of each image be if represented using a multidimensional array?\n",
"\n",
"Images are typically stored in multidimensional arrays as (height, width, channels), so here it would be (576, 768, 3).\n",
"That is, if we assume '768 by 576 pixels' means 768 horizontal pixels.\n",
"\n",
"* (Advanced) Is there enough information in these two sentences to allow you to view the images?\n",
"\n",
"No, there isn't.\n",
"We don't know the order that pixels are stored in the binary representation.\n",
"For example, they could start with a particular pixel location and then have the 3 bytes for that pixel's colour before moving onto the next location, or they could start with a colour channel and then have all the bytes for the different locations before moving onto the next colour channel."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Random noise images"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For an upcoming experiment, you would like to present images of faces for a very short duration.\n",
"However, you are finding that there is a perceptual 'afterimage' that is lingering after the image disappears.\n",
"You would like to generate an image of coloured random noise that you can display after the face, in an attempt to disrupt such an afterimage.\n",
"\n",
"Your task is to generate the random noise image and display it in the notebook.\n",
"The image is to be 96 pixels horizontally and 64 pixels vertically, and each pixel in each colour channel (RGB) is to be drawn at random from the integers 0 to 255, inclusive."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Hint"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* Think about the `size` of the array that you want to produce.\n",
"* Consider using `np.random.randint`."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true,
"scrolled": false
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"\n",
"import numpy as np\n",
"\n",
"img = np.random.randint(low=0, high=256, size=(64, 96, 3), dtype=np.uint8)\n",
"\n",
"plt.imshow(img);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Feedback sounds"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You would like to indicate to a participant whether their response was correct or incorrect by playing different sounds.\n",
"A correct response is to be followed by a 'beep' and an incorrect sound is to be followed by a 'boop'.\n",
"\n",
"Your task is to generate the two sounds, with a notebook player for each.\n",
"They are both to be pure tones of 200ms duration, sampled at 44100 Hz, and single channel (mono rather than stereo).\n",
"The 'beep' is 1000 Hz and the 'boop' is 500 Hz."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# here is a start\n",
"\n",
"import numpy as np\n",
"\n",
"import IPython.display\n",
"\n",
"sample_rate = 44100\n",
"duration_s = 0.2\n",
"\n",
"num_samples = int(sample_rate * duration_s)\n",
"\n",
"# the rest of your solution goes here; use as many cells as you need"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Hint"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* Consider starting by making an array that indicates the time (in seconds) of each sample; so here it will vary between 0 and 0.2\n",
"* Then, use the same principles we have covered previously to calculate the argument to the `sin` function."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"import IPython.display"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"sample_rate = 44100\n",
"duration_s = 0.2\n",
"\n",
"num_samples = int(sample_rate * duration_s)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"t = np.linspace(start=0.0, stop=duration_s, num=num_samples)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"beep_freq = 1000.0\n",
"boop_freq = 500.0"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"beep = np.sin(t * 2.0 * np.pi * beep_freq)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"player = IPython.display.Audio(data=beep, rate=sample_rate)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"player"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"boop = np.sin(t * 2.0 * np.pi * boop_freq)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"player = IPython.display.Audio(data=boop, rate=sample_rate)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true,
"scrolled": true
},
"outputs": [],
"source": [
"player"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Alternative feedback sounds"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When piloting, you realised that a 'boop' isn't sending a clear enough signal to the participants that their response was incorrect.\n",
"You decide to instead use 'white noise'.\n",
"\n",
"Your task is to generate such a white noise sound.\n",
"It is again 200ms in duration and mono, but now the samples are to be drawn from a uniform distribution (see `np.random.uniform`) between -0.1 and +0.1."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Hint"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* Inspect the help for `np.random.uniform` and think about what arguments you would need to supply to the parameters to meet the requirements."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"import IPython.display"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"sample_rate = 44100\n",
"duration_s = 0.2\n",
"\n",
"num_samples = int(sample_rate * duration_s)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"noise = np.random.uniform(low=-0.1, high=+0.1, size=num_samples)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"# note that, depending on the version of `IPython`, this player may\n",
"# automatically (annoyingly!) 'normalise' the waveform amplitude\n",
"player = IPython.display.Audio(data=noise, rate=sample_rate)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"player"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Data exclusions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You are running an experiment in which you are collecting response times, and you have the data from one participant in an array called `rt_data`.\n",
"Your goal is to use the calculate the mean response time across the set of trials.\n",
"\n",
"However, you have noticed that some of the values in the array are `999.0`.\n",
"After speaking to your supervisor, you learn that some trials were skipped and the response times on those trials was given a value of `999.0` to indicate this.\n",
"\n",
"You want to be able to identify the 'skipped' trials, so that you can ignore them in your mean calculation.\n",
"\n",
"* What is a potential pitfall in identifying those values in the array that equal `999.0`?\n",
"* What is a 'safe' approach for identifying the 'skipped' trials?\n",
"* How many skipped trials are there in `rt_data`?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# don't worry about understanding this cell - it reads the example data and loads it into an array\n",
"!curl https://webutils.psy.unsw.edu.au/internship_coding/notebooks/scenarios/rt_data.txt -o rt_data.txt > /dev/null 2>&1\n",
"rt_data = np.loadtxt(\"rt_data.txt\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Hint"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* Think about one of the potentially surprising aspects of how computers represent non-integer numbers."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* What is a potential pitfall in identifying those values in the array that equal `999.0`?\n",
"\n",
"The danger is that the values might not be *exactly* `999.0`, so a simple test of equality (using `==`) might be misleading.\n",
"Note that it isn't in this instance—it produces the correct output.\n",
"However, the goal here is to identify *potential* problems.\n",
"\n",
"* What is a 'safe' approach for identifying the 'skipped' trials?\n",
"\n",
"A safer approach is to use a function that is aware of such issues, such as `np.isclose`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"np.isclose(rt_data, 999.0)"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* How many skipped trials are there in `rt_data`?\n",
"\n",
"We can see from the above that there are 5 `True` entries.\n",
"Via code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"print(sum(np.isclose(rt_data, 999.0)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Analysis and visualisation of a within-subjects experiment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You have run a within-subjects experiment with two conditions, with 30 participants.\n",
"You have the data stored in the `ws_data` variable.\n",
"\n",
"Your tasks are to:\n",
"* Run a paired-sample $t$-test on the data.\n",
"* Produce a 'scatter plot', where each point shows a single participant's score on both conditions.\n",
"* Advanced: add a 'line of equality' to the plot; that is, a line that indicates where the points would lie if both the conditions were equal."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# don't worry about understanding this cell - it reads the example data and loads it into an array\n",
"!curl https://webutils.psy.unsw.edu.au/internship_coding/notebooks/scenarios/ws_data.txt -o ws_data.txt > /dev/null 2>&1\n",
"ws_data = np.loadtxt(\"ws_data.txt\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Hints"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"* Inspect the help for `scipy.stats.ttest_rel` for the paired-sample $t$ test.\n",
"* Review the end of the 'images' lesson for information on scatter plots.\n",
"* For the 'line of equality', think about the (x, y) coordinates of the two endpoints of the line. Then, use the two $x$ values as the first argument to `plt.plot` and the two $y$ values as the second argument to `plt.plot`."
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"import scipy.stats\n",
"(t, p) = scipy.stats.ttest_rel(a=ws_data[:, 0], b=ws_data[:, 1])\n",
"print(t, p)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"plt.figure();\n",
"plt.scatter(ws_data[:, 0], ws_data[:, 1]);\n",
"plt.plot([-2.5, 2.5], [-2.5, 2.5]);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Effect of optional stopping on $p$ values (advanced)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You are planning on running a study where you will ask two different groups to rate their current wellbeing on a free scale (i.e., they can give any number).\n",
"You aren't sure how many participants to run, so you are thinking of running 25 participants per group and then running an independent-samples $t$-test to test the hypothesis that the group means are different.\n",
"You figure that if such a test is not significant, you will run another 25 participants per group and then test again with all of the data (50 participants per group).\n",
"\n",
"You have a vague inkling that this might affect your false positive rate, but you aren't sure whether it does or by how much (if it does).\n",
"\n",
"Your task is to run such a simulation to identify the false positive rate when the null hypothesis is true and the planned 'optional stopping' strategy is applied.\n",
"That is, how many of 10,000 simulated experiments will be declared as statistically significant (at $\\alpha = 0.05$) when each simulated experiment could have a second round of data collection?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that there are two new Python aspects that would be useful in a solution."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, note that `if` statements can be followed by `else` statements.\n",
"The statements indented after the `else` statement are executed if the boolean following the `if` is `False`.\n",
"\n",
"For example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"p = 0.4\n",
"\n",
"if p < 0.05:\n",
" print(\"Significant\")\n",
"else:\n",
" print(\"Not significant\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Second, the `np.append` function can be used to extend an array with new elements.\n",
"\n",
"For example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"test_array = np.random.uniform(low=0, high=1, size=3)\n",
"print(test_array)\n",
"\n",
"test_array = np.append(test_array, np.ones(2))\n",
"print(test_array)"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"##### Potential solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import scipy.stats\n",
"\n",
"# null is true, so groups have the same mean\n",
"group_a_mean = 50.0\n",
"group_b_mean = 50.0\n",
"group_sd = 10.0\n",
"\n",
"n_per_set = 25\n",
"\n",
"n_sims = 10000\n",
"\n",
"n_significant = 0\n",
"\n",
"for i_sim in range(n_sims):\n",
" \n",
" group_a_data = np.random.normal(loc=group_a_mean, scale=group_sd, size=n_per_set)\n",
" group_b_data = np.random.normal(loc=group_b_mean, scale=group_sd, size=n_per_set)\n",
" \n",
" (t, p) = scipy.stats.ttest_ind(a=group_a_data, b=group_b_data)\n",
" \n",
" if p < 0.05:\n",
" n_significant += 1\n",
" \n",
" else:\n",
" # if not significant, collect a new set of data and combine it with the old set\n",
" group_a_data = np.append(group_a_data, np.random.normal(loc=group_a_mean, scale=group_sd, size=n_per_set))\n",
" group_b_data = np.append(group_b_data, np.random.normal(loc=group_b_mean, scale=group_sd, size=n_per_set))\n",
" \n",
" (t, p) = scipy.stats.ttest_ind(a=group_a_data, b=group_b_data)\n",
" \n",
" if p < 0.05:\n",
" n_significant += 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"n_significant / n_sims"
]
}
],
"metadata": {
"hide_input": false,
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": false,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": true,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": true
}
},
"nbformat": 4,
"nbformat_minor": 2
}