KRITIK JURNAL NASIONAL Judul Volume : Proses Berpikir Kreatif berdasarkan Wallas Model dalam Memecahkan Masalah Matematika : 1. Hal. 177-184 Penulis : Hevy Risqi Maharani, Sukestiyarno, Budi Waluya Sumber : International Journal on Emerging Mathematics Education (IJEME) Pendahuluan Jurnal yang berjudul “Proses Berpikir Kreatif berdasarkan Wallas Model dalam Memecahkan Masalah Matematika” merupakan penelitian dari Hevy Risqi Maharani, Sukestiyarno, dan Budi Waluya pada tahun 2017. Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang mengembangkan proses berfikir kreatif, masalah matematika dan metodenya. The Problem (masalah ) 1. Latar belakang masalah dalam jurnal ini yaitu Suatu hal yang sulit bagi guru SMP adalah memposisikan dan mengembangkan anak didiknya yang masih pada masa transisi dalam berpikir kreatif. 2. Masalah dalam jurnal ini Pada kenyataannya, yang sering terjadi dalam pembelajaran matematika adalah siswa diberi masalah tertutup untuk dipecahkan. Siswa tidak diajar untuk menggunakan pemikiran dan penalaran yang berbeda yang penting bagi mereka untuk memahami konteksnya sehingga mereka memberikan respons positif dan berpartisipasi aktif dalam proses pembelajaran. The Design 1. Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya yaitu memperkenalkan Salah satu model proses berpikir kreatif disajikan oleh Wallas (2014). Model ini terdiri dari empat tahap berbeda: persiapan, inkubasi, iluminasi, dan verifikasi.. The Prosedure 1. Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Meskipun peneliti hanya menampilkan gambar/foto dari hasil tes. 2. Kemudian terdapat tabel yang mencakup kegiatan-kegiatan siswa selama menggunakan model wallas ini. The Measurement Pada bagian teori memang sudah dijelaskan secara teorinya, namun saya lihat tidak ada sesuatu angka atau apapun yang dihitung. Mungkin saya kira karena penelitian ini adalah deskriptif kualitatif. The Interpretation 1. Kesimpulan dan hasil penelitian dalam jurnal ini konsisten sehingga pembaca dapat memahaminya secara jelas 2. Namun dengan menggunakan model Wallas masih tetap tidak bisa mengatasi dalam kasus siswa pada tingkat berpikir kreatif rendah dan sedang. . KRITIK JURNAL NASIONAL Judul : Development of Student Worksheets to Improve the Ability of Mathematical Problem Posing Volume : 1. Hal. 1-10 Penulis : Harry Dwi Putra, Tatang Herman, Utari Sumarmo Sumber : International Journal on Emerging Mathematics Education (IJEME) Pendahuluan Jurnal yang berjudul “Pengembangan Lembar Kerja Siswa untuk Meningkatkan Kemampuan Posing Masalah Matematika” merupakan penelitian dari Harry Dwi Putra, Tatang Herman, Utari Sumarmo (2017). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris dan Bahasa Indonesia, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang lembar kerja siswa, pendekatan ilmiah, bagaimana jika bukan strategi, masalah matematika. The Problem (Masalah) 3. Latar belakang masalah dalam jurnal ini yaitu Bagaimana mengembangkan lembar kerja siswa melalui pendekatan ilmiah dengan bagaimana jika tidak strategi untuk meningkatkan kemampuan siswa dalam masalah matematika 4. Namun qpa dampak dari penggunaan lembar kerja siswa melalui pendekatan ilmiah dengan bagaimana jika tidak strategi pada kemampuan siswa dalam masalah matematika The Design 3. Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan yaitu dengan literature lembar kerja siswa untuk dikembangkan agar dapat menaikkan tingkat kemampuan posing siswa dalam menyelesaikan masalah matematika. The procedure 1. Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. 2. Meskipun akan lebih rapih bila diurutkan dengan angka/huruf untuk menjdabarkan perlangkahnya. The Measurment 1. Pada bagian perhitungan terlihat cukup jelas, namun ada beberapa bagian yang agak sulit dimengerti pada rumus yang digunakan. 2. Namun tabel yang digunakan sudah cukup jelas untuk melihat hasil angkanya. The interpretation 1. Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas, namun pada bagian ini hanya sedikit yang dibahas dari hasil penelitian yaitu langsung ke point/inti hasil dilapangan. 2. Hasil penelitian pada bagian diskusi sangat jelas untuk diargumentasikan. KRITIK JURNAL NASIONAL Judul : Penilaian Kapasitas Berpikir Logis pada Siswa Kelas Tujuh (ASSESSMENT OF SEVENTH GRADE STUDENTS’ CAPACITY OF LOGICAL THINKING) Volume : 1. Hal. 75-80 Penulis : M. Fadiana, S. M. Amin , A. Lukito , A. Wardhono , S. Aishah Sumber : Jurnal Pendidikan IPA Indonesia Pendahuluan Jurnal yang berjudul “Penilaian Kapasitas Berpikir Logis pada Siswa Kelas Tujuh” merupakan penelitian dari M. Fadiana, S. M. Amin , A. Lukito , A. Wardhono , S. Aishah (2019). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang berpikir logis, operasional konkret, operasional formal dan perilaku kognitif siswa sekolah menengah. The Problem (masalah ) 4. Latar belakang masalah dalam jurnal ini yaitu peneliti sering menemukan bahwa guru jarang mengukur keterampilan berpikir logis siswa sebelum merancang strategi pembelajaran yang akan dilakukan. 5. Apa perbedaan mendasar yang ditemukan antara siswa operasional formal dan siswa transisi. The Design 2. Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya memperkenalkan Instrumen yang digunakan dalam penelitian ini adalah Logical Thinking Test (LTT) yang direvisi dari Group Assessment of Logical Thinking (GALT). GALT diciptakan untuk mengukur enam jenis penalaran yang berbeda; yaitu penalaran konservasional, penalaran proporsional, variabel kontrol, penalaran kombinatorial, penalaran probabilistik, dan penalaran korelasional (Roadrangka et al., 1983). The Prosedure 5. Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Meskipun peneliti hanya menampilkan gambar/foto dari hasil tes. 6. Kemudian terdapat tabel yang mencakup beberapa hal terkait hasil tes dan perbandingan siswa. The Measurement Pada bagian perhitungan terlihat sangat baik dan rapih untuk pembaca. Namun tak ada gambar atau lampiran/foto contoh survei yang digunakan, agar menjadi referensi juga bagi peneliti lain. The Interpretation 3. Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas, namun kata-kata yang disajikan masih kurang rapih dan agak sulit dipahami sehingga pembaca kurang memahami jelas 4. Paragraph artikel akan mudah dibaca bila tidak dibagi jadi 2 kolom perhalaman. KRITIK JURNAL NASIONAL Judul : Using the 5E Learning Cycle with Metacognitive Technique to Enhance Students’ Mathematical Critical Thinking Skills Volume : 1 No. 1 Hal. 87-98 Penulis : Runisah, Tatang Herman, Jarnawi Afgani Dahlan Sumber : International Journal on Emerging Mathematics Education (IJEME) Pendahuluan Jurnal yang berjudul “Menggunakan Siklus Belajar 5E dengan Teknik Metakognitif untuk Meningkatkan Keterampilan Berpikir Kritis Matematis Siswa” merupakan penelitian dari Runisah, Tatang Herman, Jarnawi Afgani Dahlan (2017). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang keterampilan berpikir kritis matematika, siklus belajar 5E, teknik metakognitif. The Problem Latar belakang masalah dalam jurnal ini yaitu Hasil Tren dalam Studi Matematika dan Sains Internasional (TIMSS) pada tahun 2007 dan 2011 menunjukkan bahwa skor rata-rata pencapaian mata pelajaran matematika pada tahun 2011 adalah 38 th peringkat dari 42 negara yang berpartisipasi. Salah satu faktor yang mempengaruhi kurangnya kemampuan berpikir kritis matematis siswa adalah proses pembelajaran. Di Indonesia, praktik mengajar difokuskan pada konten materi dan mengabaikan pengembangan kemampuan berpikir siswa (Rohaeti, 2010). Dari hasil inkuiri Balitbang Depdiknas pada 2007, implementasi pembelajaran di Indonesia umumnya masih menggunakan metode ceramah dan tanya jawab (Balitbang Depdiknas, 2007). Berbagai masalah dalam aspek kehidupan terjadi pada 21 st abad. Untuk mengatasi masalah ini, keterampilan berpikir kritis diperlukan. The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya mempekenalkan 5E Learning Cycle (LC) dengan Metacognitive technique (LCM) adalah model pembelajaran yang mengintegrasikan langsung teknik Metacognitive di setiap tahap LC. LC memiliki lima tahap yaitu: terlibat, mengeksplorasi, menjelaskan, menguraikan dan mengevaluasi. metakognisi adalah kesadaran seseorang tentang proses kognitif dan kemandirian untuk mencapai hasil. The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Secara keseluruhan sudah cukup bagus.. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas. Hasil data yang diberikan sudah memuat jawaban dari masalah-masalah yang dcari. KRITIK JURNAL NASIONAL Judul : PROSPECTIVE MATHEMATICS TEACHERS’ COGNITIVE COMPETENCIES ON REALISTIC MATHEMATICS EDUCATION Volume : 11. No. 1 Hal. 17-44 Penulis : Rezan Yilmaz Sumber : Journal on Mathematics Education (JME) Pendahuluan Jurnal yang berjudul “KOMPETENSI KOGNITIF GURU MATEMATIKA PROSPEKTIF DALAM PENDIDIKAN MATEMATIKA REALISTIS” merupakan penelitian dari Rezan Yilmaz (2020). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang Pendidikan Matematika Realistik, Calon Guru Matematika, Kompetensi Kognitif, Masalah Kontekstual. The Problem Dalam hal ini, studi terutama berfokus pada RME dalam hal pendekatan dan jawaban untuk pertanyaan "Bagaimana kognitifnya kompetensi calon guru matematika terkait dengan RME? "telah dicari. Dengan demikian, submasalah "Bagaimana calon guru matematika menjelaskan RME dan implementasinya? Bagaimana Apakah mereka menafsirkan persamaan dan perbedaan dengan pendekatan lain? Bagaimana masalah kontekstual lakukan mereka berpose sesuai dengan struktur teoritis RME? ". The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya. Desain penelitian adalah studi kasus, yang telah dievaluasi sebagai kasus RME The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Secara keseluruhan sudah cukup bagus. Tahap-tahap teknis dijalankan secara beurut dan rapih, namun untuk RME yang digunakan masih pada tahap yang biasa atau mainstream. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas. Hasil data yang diberikan sudah memuat jawaban dari masalah-masalah yang dcari. Namun, masih diperlukan RME dikoneksikan ke internet atau sosmed yang relevan untuk mencapai pembelajaran yang bermakna dalam matematika. KRITIK JURNAL INTERNASIONAL Judul : ELEMENTARY PRESERVICE TEACHERS’ KNOWLEDGE, PERCEPTIONS AND ATTITUDES TOWARDS FRACTIONS: A MIXED-ANALYSIS Volume : 11 No. 1 Hal. 59-76 Penulis : Roslinda Rosli, Dianne Goldsby , Anthony J. Onwuegbuzie , Mary Margaret Capraro , Robert M. Capraro , Elsa Gonzalez Y. Gonzalez Sumber : Journal on Mathematics Education (JME) Pendahuluan Jurnal yang berjudul “PENGETAHUAN PENGAJARAN, PERSEPSI DAN SIKAP GURU SD TERHADAP PECAHAN: ANALISIS CAMPURAN” merupakan penelitian dari Roslinda Rosli, Dianne Goldsby , Anthony J. Onwuegbuzie , Mary Margaret Capraro , Robert M. Capraro , Elsa Gonzalez Y. Gonzalez (2020). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang Sekolah Dasar, Masalah posing, Program Persiapan Guru, Calon Guru, Metode Campuran. The Problem (masalah ) Latar belakang masalah dalam jurnal ini yaitu Sejumlah literatur telah mendokumentasikan bahwa mayoritas guru dalam jabatan dan guru pra-jabatan memiliki pengetahuan mendalam yang terbatas untuk mengajar matematika. Terlepas dari bukti ini, sedikit yang diketahui tentang sifat pengetahuan untuk mengajar fraksi, persepsi dan sikap yang harus dimiliki guru dipelihara selama persiapan guru, tempat di mana guru seharusnya mendapatkan pengajaran mereka. The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya. Dalam studi ini, peneliti fokus pada mengukur integrasi modul Koneksi Matematika TEKS dengan model pembelajaran beton praktik dalam memfasilitasi konstruksi pengetahuan guru pra-layanan, persepsi dan sikap terhadap pecahan selama program persiapan guru. The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Meskipun peneliti hanya menampilkan data secara penuh tulisan, tanpa ada bantuan gambar/foto maupun diagram. The Measurement Pada bagian perhitungan terlihat sangat baik, namun tidak rapih untuk pembaca. Tak ada gambar atau lampiran/foto contoh survei yang digunakan, agar menjadi referensi juga bagi peneliti lain. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas, namun kata-kata yang disajikan masih kurang rapih dan agak sulit dipahami sehingga pembaca kurang memahami jelas. Masih butuh penambahan data dari beberapa instansi terkait subjek penelitian untuk melengkapi hasil data yang disimpulkan. KRITIK JURNAL INTERNASIONAL Judul : The Relation Between Reinforcement Learning Parameters And The Influence Of Reinforcement History On Choice Behavior Volume : 66. Hal. 59-69 Penulis : Kentaro Katahira Sumber : Journal of Mathematical Psychology Pendahuluan Jurnal yang berjudul “Hubungan Antara Parameter Pembelajaran Penguatan Dan Pengaruh Sejarah Penguatan Pada Perilaku Pilihan” merupakan penelitian dari Kentaro Katahira (2015). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang Penguatan pembelajaran, Sejarah ketergantungan, Model regresi, dan Analisis berbasis model. The Problem Dalam hal ini, studi terutama berfokus pada menyelidiki hubungan antara parameter model RL dan model regresi akan memberikan informasi berharga tentang faktor perilaku mana yang mungkin mendasari perbedaan dalam parameter model. Sebaliknya, dengan menggunakan relasi, orang dapat memprediksi jenis urutan perilaku mana yang dapat diharapkan diberikan seperangkat parameter model tertentu. The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya memperkenalkan Model Reinforcement learning (RL) yang telah banyak digunakan untuk menganalisis perilaku pilihan manusia dan hewan lain dalam berbagai bidang, termasuk psikologi dan ilmu saraf. Model berbasis regresi linier yang secara eksplisit mewakili bagaimana hadiah dan sejarah pilihan mempengaruhi pilihan masa depan juga telah digunakan untuk memodelkan perilaku pilihan The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Secara keseluruhan sudah cukup bagus. Tahap-tahap teknis dijalankan secara beurut dan rapih, namun untuk Untuk mencapai tujuan ini, diperlukan perhitungan analitik dan simulasi numerik karena terdapat materi tugas-tugas probalitas. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas. Hasil data yang diberikan sudah memuat jawaban dari masalah-masalah yang dicari. Namun, masih belum bisa menentukan kebiasaan seseorang atau siswa dalam membuat pilihan. KRITIK JURNAL INTERNASIONAL Judul : An Analysis for the Qualitative Improvement of Education and Learning based on the Way of Learner Errors in Descriptive Questions Volume : 15. No. 3 Penulis : Michiko Tsubaki, Wataru Ogawara, Kenta Tanaka Sumber : INTERNATIONAL ELECTRONIC JOURNAL OF MATHEMATICS EDUCATION Pendahuluan Jurnal yang berjudul “Suatu Analisis untuk Peningkatan Kualitatif Pendidikan dan Pembelajaran berdasarkan Kesalahan Pelajar dalam Pertanyaan Deskriptif” merupakan penelitian dari Michiko Tsubaki, Wataru Ogawara, dan Kenta Tanaka (2020). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang pembelajaran dan pendidikan dalam probabilitas dan statistik, pertanyaan deskriptif, ekstraksi karakteristik jawaban, analisis kesalahan, analisis jaringan Bayesian, dan penalaran probabilistic. The Problem Dalam hal ini, menyelidiki hal yang telah disimpulkan oleh Kahneman dan Tversky (1982), Sulistyani (2019) menunjukkan bahwa ada 4 tahap kesalahan siswa dalam statistik inferensial. 1) Kesalahan dalam pemahaman terjadi karena siswa tidak dapat membaca tabel statistik atau membaca output dalam pertanyaan. 2) Kesalahan transformasi terjadi karena siswa tidak tepat dalam menerapkan / memilih jenis statistik uji yang digunakan atau menulis hipotesis. 3) Kesalahan keterampilan proses terjadi karena siswa kurang berhati-hati dalam menghitung dan ketidakmampuan untuk menginterpretasikan hasil perhitungan. 4) Kesalahan dalam tahap pengkodean terjadi karena siswa tidak menjawab dengan benar atau tidak tepat dalam menarik kesimpulan dalam pengujian hipotesis. Hasil-hasil ini menunjuk pada sifat-sifat yang khas pada isi pembelajaran di bidang probabilitas dan statistik, menunjukkan kecenderungan yang mungkin dalam cara para pelajar probabilitas dan statistik membuat kesalahan. The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya. Penelitian ini mengusulkan ekstraksi karakteristik jawaban sebagai variabel, berdasarkan pada bentuk kesalahan yang menjadi ciri perbedaan dalam tingkat pemahaman peserta didik menggunakan pertanyaan deskriptif lengkap terkait dengan probabilitas dan statistik. Kemudian, dengan menggunakan variabel-variabel ini, kami melakukan analisis pohon regresi pada pemahaman peserta didik dari konten penelitian untuk setiap masalah, serta analisis bagan karakteristik item untuk memperkirakan pemahaman umum, dan analisis relasional antara pemahaman dan strategi belajar menggunakan Jaringan Bayesian. Dengan demikian, selanjutnya kami mengusulkan metode untuk mengekstraksi kebijakan pembelajaran yang efektif atau strategi pembelajaran untuk peserta didik secara keseluruhan, serta peserta didik individu, melalui penalaran probabilistik berdasarkan jaringan Bayesian. The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Secara keseluruhan sudah cukup bagus. Tahap-tahap teknis dijalankan secara beurut dan rapih, namun untuk Untuk mencapai tujuan ini, diperlukan perhitungan analitik dan simulasi numerik karena terdapat materi tugas-tugas probalitas. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas. Hasil data yang diberikan sudah memuat jawaban dari masalah-masalah yang dcari. Namun, disarankan adanya strategi pembelajaran yang efektif secara terpisah untuk setiap konsep dalam setiap pertanyaan, serta untuk setiap tingkat keahlian, apakah itu dasar (memahami definisi, dll.) Atau diterapkan (Sadar objek derivasi, deskripsi lengkap, dll. ), dalam karakteristik jawaban. KRITIK JURNAL INTERNASIONAL Judul : A SPARSE LATENT CLASS MODEL FOR COGNITIVE DIAGNOSIS Volume : 85. No. 1 Hal. 121-153 Penulis : Yinyin Chen, Steven Culpepper, dan Feng Liang Sumber : Psychometrika Pendahuluan Jurnal yang berjudul “MODEL KELAS LATEN YANG JARANG UNTUK DIAGNOSIS KOGNITIF” merupakan penelitian dari Yinyin Chen, Steven Culpepper, dan Feng Liang (2020). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang model kelas laten jarang, pemilihan variabel Bayesian, dan identifikasi. The Problem Dalam hal ini, menyelidiki hal yang telah disimpulkan pra-menentukan CDM yang tepat dapat sulit dalam praktiknya, terutama ketika tidak ada pengetahuan sebelumnya tentang tes yang tersedia. Lebih lanjut, ada kemungkinan bahwa berbagai pertanyaan (item) dalam satu pengujian perlu dimodelkan oleh CDM yang berbeda. Misalnya, tes matematika dapat mencakup pertanyaan yang dapat diselesaikan dengan menggunakan keterampilan yang berbeda, yang menyiratkan model disjungtif The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuannya memperkenalkan Cognitive diagnostic models (CDMs) adalah variabel laten yang dikembangkan untuk menyimpulkan keterampilan laten, pengetahuan, atau kepribadian yang mendasari respons terhadap tes dan tindakan pendidikan, psikologis, dan ilmu sosial. Penelitian terbaru berfokus pada teori dan metode untuk menggunakan model kelas laten jarang (SLCMs) dalam cara eksplorasi untuk menyimpulkan proses laten dan struktur respon yang mendasarinya. The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Secara keseluruhan sudah cukup bagus. Tahap-tahap teknis dijalankan secara beurut dan rapih, namun Kontribusi penting untuk praktik adalah bahwa kondisi pengidentifikasian generik baru kami lebih cenderung terpenuhi dalam aplikasi empiris daripada kondisi yang ada yang memastikan pengidentifikasian yang ketat. Mempelajari struktur laten yang mendasarinya dapat dirumuskan sebagai masalah pemilihan variabel. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas. Hasil data yang diberikan sudah memuat jawaban dari masalah-masalah yang dicari. Namun, pada bagian diskusi belum diberikan kesimpulan yang lebih kongkret lagi. KRITIK JURNAL INTERNASIONAL Judul : Teachers’ Real and Perceived of ICTs Supported-Situation for Mathematics Teaching and Learning Volume : 1. No. 1 Hal. 11-24 Penulis : Maman Fathurrohman, Anne L. Porter, Annette L. Worthy Sumber : International Journal on Emerging Mathematics Education (IJEME) Pendahuluan Jurnal yang berjudul “Keadaan Sebenarnya dan Persepsi Guru pada dukungan TIK-Situasi untuk Pengajaran dan Pembelajaran Matematika” merupakan penelitian dari Maman Fathurrohman, Anne L. Porter, Annette L. Worthy (2017). Isi jurnal terdiri atas judul, nama penulis, abstrak dalam Bahasa inggris, kata kunci, pendahuluan, perangkat penelitian, metode yang di gunakan, teori dan contoh, penutup dan daftar pustaka. Secara garis besar jurnal tersebut membahas tentang survai, pembelajaran berbasis teknologi, TIK, infrastruktur, fasilitas, dan sumber daya. The Problem Di bidang pendidikan, teknologi telah diadopsi sebagaimana dapat dilihat dalam kegiatan akademik sehari-hari, tidak hanya di universitas tetapi juga sekolah menengah dan dasar di Indonesia. Mengajar dan belajar dengan teknologi telah terjadi di banyak tempat karena manfaatnya memberikan keuntungan dalam membuat proses belajar mengajar lebih efektif dan efisien. Satu hal penting yang perlu dipertimbangkan adalah perspektif guru terkait komponen-komponen TIK. The Design Pembahasan maupun isi dalam jurnal ini sudah sesuai dalam menjawab permasalahan yang disampaikan serta tujuanny. Metode yang digunakan adalah survai. Instrumen yang digunakan adalah kuesioner, pedoman wawancara tidak terstruktur, dan handycam. The Prosedure Pada jurnal sudah di sebutkan beberapa bahan, tahap dan pengujiannya. Secara keseluruhan sudah cukup bagus. Meskipun latar belakang pendidikan resmi guru baik, pelatihan mereka, menghadiri atau berpartisipasi dalam pelatihan terkait TIK masih rendah. The Interpretation Kesimpulan dan hasil penelitian dalam jurnal ini cukup jelas. Hasil data yang diberikan sudah memuat jawaban dari masalah-masalah yang dicari. Namun, mungkin bisa disebutkan komponen yang paling dibutuhkan dan lebih efektif pada kesimpulan. psychometrika https://doi.org/10.1007/s11336-019-09693-2 A SPARSE LATENT CLASS MODEL FOR COGNITIVE DIAGNOSIS Yinyin Chen UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Steven Culpepper UNIVERSITY OF ILLINOIS AT URBANA–CHAMPAIGN Feng Liang UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Cognitive diagnostic models (CDMs) are latent variable models developed to infer latent skills, knowledge, or personalities that underlie responses to educational, psychological, and social science tests and measures. Recent research focused on theory and methods for using sparse latent class models (SLCMs) in an exploratory fashion to infer the latent processes and structure underlying responses. We report new theoretical results about sufficient conditions for generic identifiability of SLCM parameters. An important contribution for practice is that our new generic identifiability conditions are more likely to be satisfied in empirical applications than existing conditions that ensure strict identifiability. Learning the underlying latent structure can be formulated as a variable selection problem. We develop a new Bayesian variable selection algorithm that explicitly enforces generic identifiability conditions and monotonicity of item response functions to ensure valid posterior inference. We present Monte Carlo simulation results to support accurate inferences and discuss the implications of our findings for future SLCM research and educational testing. Key words: sparse latent class models, Bayesian variable selection, identifiability. 1. Introduction Cognitive diagnostic models (CDMs) are latent class models developed for the inference of educational, psychological, and social science tests. In CDMs, the latent variables are often defined as skills, knowledge, or personalities needed by a subject to solve a given test item. Consider a test that consists of J items and involves K skills. The observable response Y = (Y1 , . . . , Y J ) for a subject is a binary random vector, indicating the correctness of the subject’s answers to the J items. The latent class of a subject is indexed by a K -dimensional binary vector α, which are referred to as an attribute profile, which suggests the mastery of each skill. Given α, Y is modeled by a product of J independent Bernoulli random variables with parameter θ j,α = P(Y j = 1|α). In many CDMs, θ j,α depends on a K -dimensional binary vector q j , where element q jk = 1 if attribute k is relevant to item j and zero otherwise. The relevant skills to all items are usually presented by a J × K matrix, Q = [q 1 . . . , q J ]T , called the Q-matrix. CDMs provide a statistical framework to identify relevant attributes of test items and to classify the mastery/non-mastery of test subjects on those attributes, which provide useful insights for researchers and educators. Various CDMs have been proposed in the literature and they differ from each other in their assumptions on how θ j,α depends on the Q-matrix. For example, the Correspondence should be made to Steven Culpepper, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana–Champaign, 725 South Wright Street, Champaign, IL 61820, USA. Email: [email protected] © 2020 The Psychometric Society PSYCHOMETRIKA DINA (Deterministic Input, Noisy ‘And’ gate) model (Haertel 1989; Junker and Sijtsma 2001), the generalized DINA model (de la Torre 2011) and the reduced reparameterized unified (rRUM) model (Hartz 2002; Rupp et al. 2010) are conjunctive models where all relevant skills are needed to have the highest positive response probability. On the other hand, under disjunctive models, such as the DINO (Deterministic Input, Noisy ‘Or’ gate) model (Templin and Henson 2006), at least one relevant skill is needed, whereas compensatory models (e.g., see Davier (2005), for a special case of the general diagnostic model) allow students to compensate for missing some skills by having others. However, pre-specifying an appropriate CDM can be difficult in practice, especially when no prior knowledge of the test is available. Further, it is likely that different questions (items) in a single test need to be modeled by different CDMs. For instance, a mathematics test may include questions that can be solved using different skills, which implies a disjunctive model. The same test may also include questions that involve multiple steps and require students to master all relevant skills in order to get the final correct answer, which implies a conjunctive model. For such tests, the specification of a single CDM would fail to capture the real latent patterns. To address this issue, a novel, model-free approach was proposed by Chen et al. (2015), which is based on an alternative representation of CDMs via a mixture of generalized linear models (GLMs). In particular, the jth Bernoulli parameter for latent class α is modeled as K K β j,k αk + β j,kk αk αk + · · · + β j,12...K αk P Y j = 1 α, β j = β j,0 + k=1 k>k k=1 (1) where (·) is an arbitrary cumulative distribution function (CDF) and β j is a coefficient vector to be estimated. It can be shown that all of the aforementioned CDMs are special cases of (1) with particular sparsity patterns of β j . In addition, the sparsity pattern of β j provides information about which attributes are relevant. Consequently, the estimation of Q can be reformulated as a variable selection problem involving GLMs. A fundamental issue with latent mixture models is model identifiability. Throughout, identifiability is defined up to a permutation of the K attributes. That is, we do not discuss the trivial identifiability issue due to label switching, since it is a well-understood issue and we know how to handle it in practice. The first rigorous study on the identifiability of the Q-matrix was given by Liu et al. (2013) with a focus on DINA models with known guessing parameters. Chen et al. (2015) extended the result of Liu et al. (2013) to DINA and DINO models with unknown model parameters. Xu (2017) established identifiability conditions for general CDMs when Q is known; Xu and Shang (2017) later provided identifiability conditions for general CDMs when Q is unknown. Although the results from Xu (2017) and Xu and Shang (2017) are applicable to the general model based on latent mixture of GLMs, their identifiability conditions, which require two identity matrices embedded in Q, are too strong to be satisfied in practice. Recently, Fang et al. (2019) proposed identifiability conditions in terms of the distribution of Y rather than Q, but their conditions are still stronger than the ones needed for our result. In this paper, we provide a new set of generic identifiability conditions for general CDMs. Our conditions are weaker than the ones in the aforementioned papers since those papers studied conditions for strict identifiability. As stated by Allman et al. (2009), “…generic identifiability implies that the set of points for which identifiability does not hold has measure zero," which is enough for practical data analysis. For example, Bernoulli mixtures are not strictly identifiable (Goodman 1974; Gyllenberg et al. 1994a), but given Bernoulli mixtures are generically identifiable, they often lead to valid statistical inference in practice (Carreira-Perpiñán and Renals 2000; Allman et al. 2009). Results for generic identifiability are established in Allman et al. (2009) for YINYIN CHEN ET AL. latent class models, which cannot be directly applied to CDMs, since CDMs are restricted latent class models. However, we can extend their proof technique, which is based on the tensor product framework in Kruskal (1976, 1977), to handle CDMs. For a feasible implementation, we adopt a Bayesian approach to the estimation and variable selection of (1), and develop a Gibbs sampling algorithm for computation. Different from the vanilla Gibbs algorithm for Bayesian variable selection, our Gibbs algorithm is specially designed to ensure that each posterior draw of the sparse model parameters β j ’s is from the identifiable space, while the algorithms from Chen et al. (2015), Xu and Shang (2017) and Fang et al. (2019) cannot ensure that their estimation of Q or other model parameters satisfies their identifiability conditions. In a recent work, Chen et al. (2018) also used MCMC method to stochastically search over identifiable parameter space. However, Chen et al. (2018) focus on DINA and DINO models with strict identifiability conditions, whereas we develop an MCMC sampling algorithm for general CDMs with weaker, generic identifiability conditions. The remainder of this paper is organized as follows. Section 2 introduces the setup of the sparse latent class models. Section 3 addresses identifiability issues and discusses the generic identifiability conditions. Section 4 introduces the Gibbs sampler for posterior inference. We report results from a simulation study in Sect. 5 and results from two real applications in Sect. 6, and close with a discussion in Sect. 7. Proofs and other technical details are included in “Appendix”. 2. Model and Applications 2.1. Model Setup Consider a test consisting of J items and involving K latent skills. Let Y = (Y1 , . . . , Y J ) denote the J binary responses from a subject. Based on the mastery of the K skills, each subject has an attribute profile α ∈ {0, 1} K , where αk = 1 indicates that the subject masters skill k and zero otherwise. In SLCM, given the attribute profile α, the J binary variablesY1 , . .. , Y J are modeled as independent Bernoulli variables with Bernoulli parameter θ j,α = aTα β j where aα = 1, α1 , . . . , α K , α1 α2 , . . . , α K −1 α K , . . . , K T αk (2) k=1 is a 2 K -dimensional alternative representation of the binary vector α with aTα β j = β j,0 + K k=1 β j,k αk + k>k β j,kk αk αk + · · · + β j,12...K K αk . (3) k=1 The regression coefficients β j form a sparse vector, in which the nonzero elements represent the effects of skills or combinations of skills on the response of item j. In particular, where coefficients in (3) are nonzero imply the dependence between item j and the K skills. To identify which main effects and/or interactions of the K skills are relevant to item j, we introduce a 2 K -dimensional binary structure vector K δ j = (δ j,0 , δ j,1 , . . . , δ j,K , δ j,12 , . . . , δ j,(K −1)K , . . . , δ j,1...K )T ∈ {0, 1}2 , PSYCHOMETRIKA Table 1. Sparse patterns of various CDMs DINA DINO G-DINA NC-RUM C-RUM δ j,0 δ j,1 δ j,2 1 1 1 1 1 1 1 1 1 1 1 1 1 δ j,3 δ j,12 1 1 1 δ j,13 δ j,23 δ j,123 Note β j,1 = β j,2 = −β j,12 (x) = x (x) = exp(x) (x) = logit−1 (x) with 1 indicating that the corresponding coefficient is active, i.e., its β value is nonzero, and 0 indicating that the coefficient is inactive, i.e., its β value is zero. The intercept β j,0 is usually assumed to be active, so δ j,0 is fixed at 1. From now on, we will use B J ×2 K , referred to as the coefficient matrix, to denote the collection of β j ’s, and J ×2 K , referred to as the sparsity matrix, to denote the collection of δ j ’s. 2.2. Sparsity Patterns Many popular CDMs can be reparameterized as special cases of SLCM with particular sparsity patterns, including the DINA model, the DINO model, the G-DINA (Generalized DINA) model, the NC-RUM (reduced noncompensatory reparameterized unified model) (DiBello et al. 1995; Rupp et al. 2010), and the C-RUM (Compensatory-RUM) (Hagenaars 1993; Maris 1999). In Table 1, we present the sparsity patterns of these aforementioned CDMs, if being reparameterized as SLCMs. For simplicity, we assume there are K = 3 latent skills, and the first two skills are relevant to each item. The detailed derivation for reparameterization of CDMs is provided in “Appendix A”. The DINA model is a conjunctive model with θ j,α taking only two possible values: one for students who have all the relevant skills, and the other for students who miss any of the relevant skills. If reparameterized as an SLCM, the DINA model is a sparse model with only one active coefficient, the highest interaction term that involves all the relevant skills, in addition to the intercept. In contrast, the DINO model is a disjunctive model, in which one value of θ j,α is for students who have at least one of the relevant skills, and the other is for students who miss all of the relevant skills. If reparameterized as an SLCM, the DINO model turns out to be a dense model with all the main effects and interactions among the relevant skills being active. In addition, there are some constraints on the coefficients: the active coefficients in odd orders are all equal and positive, while the values of those in even orders are the additive inverse of the odd ones. The G-DINA model is a generalization of the DINA model, which assumes θ j,α can be decomposed into the sum of the effects from relevant skills and their interactions. So similar to the DINO model, the G-DINA model is also a dense model with all the main effects and interactions among the relevant skills being active. The NC-RUM model assumes that missing any of the relevant skills reduces the positive response probability by a multiplicative penalty term. Xu (2017) has shown that the NC-RUM model is equivalent to a log-link additive model with just the main effects. Similar to the NC-RUM model, the C-RUM model is also an additive model involving only the main effects, but with a logit link function. An advantage of SLCM is that it contains all CDMs that admit a K -dimensional binary attribute profile, including the ones that do not fall into any of the aforementioned CDMs. Consider two sparsity patterns shown in Table 2 and assume the active coefficients are all positive. In the first case, mastering skill 1 alone increases the positive response probability, while mastering YINYIN CHEN ET AL. Table 2. Other sparse patterns of q = (1, 1, 0) δ j,0 1 1 δ j,1 δ j,2 1 1 δ j,3 δ j,12 δ j,13 δ j,23 δ j,123 1 1 skill 2 alone does not; mastering skill 2, however, increases the positive response probability conditioning on the mastery of skill 1. The second case has a similar interpretation. These two cases do not belong to any of the CDMs mentioned above, but can be modeled by SLCM. 2.3. From Q-matrix to -matrix Although the Q-matrix has been widely used for cognitive diagnostic modeling, it only provides partial information regarding the item–attributes relationship (Fang et al. 2019). Given q j , we know the relevant attributes, but it is not clear how they interact with each other to affect the positive response probability without specifying a particular CDM. In practice, it is challenging to pre-specify a CDM when no prior information of the test is available. Furthermore, it is possible that different items in a test follow different CDMs, so there is no single CDM that is appropriate for all the items in the test. In SLCM, the sparsity matrix J ×2 K , which does not require any pre-specified CDM, provides a more general and informative description of the item–attributes relationship. For example, all the CDMs listed in Tables 1 and 2 have the same q j = (1, 1, 0) but their structure vectors δ j ’s could be different. In cases, the Q-matrix is preferred as a summary of the relevant skills, we extract q j based on δ j as follows: for any attribute k, if there exists a subset of the K attributes, {k1 , k2 , . . . , kl } ⊆ {1, . . . , K }, such that, k ∈ {k1 , k2 , . . . , kl } and δ j,k1 k2 ...,kl = 1, then q jk = 1, otherwise, q jk = 0. That is, q jk = 1 if any element of δ j that is relevant to skill k is nonzero. 3. Identifiability Model identifiability is of great importance in the study of CDMs. In Statistics, a model is identifiable if it is theoretically possible to learn the true values of its underlying parameters after obtaining an infinite number of observations. Mathematically, this is equivalent to saying that different values of the parameters must correspond to different probability distributions of the observable variables. Identifiability conditions are technical restrictions, under which the model is identifiable. In this section, we establish a set of identifiability conditions of SLCM in terms of . We first introduce the identifiability issue encountered in CDMs in Sect. 3.1 and review prior research in Sect. 3.2. Then, we introduce the generic identifiability in the context of SLCM in Sect. 3.3 and propose a set of generic identifiability conditions in Sect. 3.4. The proof sketch of generic identifiability is provided in Sect. 3.5, and detailed proofs are given in “Appendix B.” 3.1. Identifiability Issue In CDMs, the observable variables are Y = (Y1 , . . . , Y J ),, and the parameters of interest are the latent class proportion vector π and the coefficient matrix B. Denote the parameter space of (π , B) by (π , B) = {(π, B) : π ∈ (π ), B ∈ (B)}, PSYCHOMETRIKA K where (π ) = {x ∈ R2 : x1 + · · · + x2 K = 1, xi > 0} is a (2 K − 1)-dimensional simplex and K (B) is the parameter space of the coefficient matrix B, which could be the whole space R J ×2 K or a subset of R J ×2 if we constrain the Q-matrix or the -matrix. Given an attribute profile/class α, the joint distribution of Y = {Y1 , . . . , Y J } is a product of Bernoulli distributions, which can be described by a J -dimensional 2 × · · · × 2 table J Pα (B) = (θ j,α , 1 − θ j,α ) j=1 where denotes the Kronecker product. The marginal distribution of Y over different classes is given by P(π , B) = πα Pα (B). α Definition 1. (Identifiable). A parameter set (π , B) ∈ (π , B) is identifiable, if P(π , B) = P(π̄ , B̄) ⇔ (π , B) ∼ (π̄, B̄), where (π̄, B̄) is another parameter set from (π , B) and “∼" means the two sets of parameters are identical up to label switching. 3.2. Prior Work CDMs are mixtures of Bernoulli distributions, which are known to be non-identifiable even if we ignore the non-identifiability due to label switching (Teicher et al. 1961; Yakowitz and Spragins,1968; Goodman 1974; Gyllenberg et al. 1994b). It is, however, possible for CDMs to be identifiable if certain restrictions on the Q-matrix and/or {θ j,α } are satisfied. For instance, as discussed in Chiu et al. (2009) and Liu et al. (2013), the completeness of Q-matrix, a condition that requires the Q-matrix to contain an identity matrix after row permutation, is believed to be a necessary condition for the identifiability of DINA models. Different identifiability conditions have been proposed for different CDMs in the literature. For example, Liu et al. (2013) studied the DINA model with complete knowledge of guessing parameters, and Chen et al. (2015) studied the DINA/DINO models. For general CDMs, Xu (2017) and Xu and Shang (2017) proposed a set of identifiability conditions in terms of the Q-matrix and the monotonicity constraints. But their conditions are too strong in practice, especially when K is large, which will be discussed in the next paragraph. Another set of identifiability conditions for general CDMs was proposed by Fang et al. (2019), which are still stronger than ours and also difficult to be incorporated in algorithms since they are expressed in terms of {θ j,α }. The monotonicity constraints imposed by Xu (2017) and Xu and Shang (2017) are as follows: min θ j,α ≥ θ j,0 , α∈S0 max θ j,α < min θ j,α = max θ j,α , α∈S0 α∈S1 (4) α∈S1 where S0 = {α|α q j , α 0} is the set of classes with at least one skill but not mastering all the relevant skills and S1 = {α|α q j } is the set of classes mastering all the relevant skills, where the YINYIN CHEN ET AL. notation “α α̃" means αk ≥ α̃k for all k = 1, . . . , K , and “α α̃" means αk ≥ α̃k for all k and there exists at least one k0 , such that αk0 > α̃k0 . Constraints (4) imply that students without any skill (i.e., α = 0) should have the lowest probability to give a positive answer to item j, students with all the relevant skills (α ∈ S1 ) should have the highest probability, and additional skills beyond the relevant skills are not expected to increase the probability. Under such constraints, the model identifiability is guaranteed (Xu 2017; Xu and Shang 2017) when the Q-matrix (after row swapping) takes the form of ⎛ ⎞ IK Q = ⎝ IK ⎠, Q (5) where I K is an identity matrix of size K and there are additional constraints on Q . However, condition (5) implies that the first 2K items degenerate into items in a DINA/DINO model with only one relevant skill. When K is relatively large, e.g., close to J/2, such a Q-matrix essentially forces the model to be a DINA/DINO model, which is contrary to the original intention of general CDMs. 3.3. Generic Identifiability Generic identifiability, which is less stringent than Definition 1, is introduced in Allman et al. (2009). Generic identifiability allows some parameter values to be non-identifiable as long as these exceptional values are of measure zero with respect to the parameter space. As pointed out in Allman et al. (2009), generic identifiability of a model is generally sufficient in practice, since one is unlikely to face the non-identifiability problem when almost all parameters (except a measure zero set) are identifiable. Allman et al. (2009) has proved that a general mixture of Bernoulli products is generically identifiable, provided that the number of items is larger than twice the number of classes. However, there is a gap between the result established in Allman et al. (2009) and SLCM K studied in this paper. The parameter space of the former is fixed, corresponding to (B) = R J ×2 in the context of SLCM, whereas the parameter space (B) of an SLCM varies with and is usually of dimension less than J × 2 K . In other words, unless = 1 J ×2 K (i.e., each coefficient in B is active), for any other , the whole parameter space of B is a measure zero subspace of K R J ×2 . So the result from Allman et al. (2009) is not applicable to SLCM, as it is meaningless to discuss measure zero subspace without a fixed parameter space. To discuss generic identifiability for SLCM, we need to first define the parameter space by taking into consideration of their sparsity structures. For a given sparsity structure , we denote its corresponding parameter space by (π , B) = {(π, B) : π ∈ (π ), B ∈ (B)}, where (B) consists of the coefficient matrices that can only have nonzero entries at positions where the corresponding elements in is 1. So it suffices to think (B) = R|| , where || denotes the total sum of the entries in . Let C denote the set of non-identifiable parameters from (π, B): C = {(π , B) : P(π , B) = P(π̃ , B̃), (π, B) ∼ (π̄, B̄), (π, B) ∈ (π , B), (π̃, B̃) ∈ ˜ (π , B)}. PSYCHOMETRIKA Note that the non-identifiability of a parameter set (π , B) ∈ C could be due to another parameter ˜ set (π̃, B̃) with a different sparsity structure . If the non-identifiable set C is of measure zero with respect to (π, B), we say (π, B) is a generically identifiable parameter space. Definition 2. (Generically identifiable). The parameter space (π , B) is generically identifiable, if the Lebesgue measure of C with respect to (π, B) is zero. To distinguish the two definitions of identifiability, in the following text, we refer to the identifiability defined in Definition 1 as strict identifiability. 3.4. Identifiability Conditions In this section, we discuss two sets of conditions for generic identifiability and strict identifiability. We start with the following two conditions needed for generic identifiability. ⎞ ⎛ D1 (G1) The true sparsity matrix takes the form of = ⎝ D2 ⎠ after row swapping, where K is a (J − 2K ) × 2 binary matrix and D1 , D2 ∈ Dg with Dg = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ D ∈ {0, 1} K ×2 K ⎡ ∗ 1 ∗ ... ∗ ... ⎢∗ ∗ 1 . . . ∗ . . . ⎢ : D = ⎢. . . .. ⎣ .. .. ∗ ∗ ∗ ... 1 ... ⎤⎫ ∗ ⎪ ⎪ ⎪ ⎬ ∗⎥ ⎥ .. ⎥ , . ⎦⎪ ⎪ ⎪ ⎭ ∗ where ∗ can be either 0 or 1. (G2) For any k = 1, 2, . . . , K , there exists a jk > 2K , such that δ jk ,k = 1 . Theorem 1 The parameter space (π , B) is generically identifiable, if conditions (G1) and (G2) are satisfied. Remark 1 Theorem 1 does not require the monotonicity constraints (4), but it remains valid if, in addition, the monotonicity constraints are imposed on B. Remark 2 Theorem 1 applies to any CDM that has a real analytic link function, including but not limited to the probit link, the logit link, the log link, and the identity link. Remark 3 If the ∗ entries in condition (G1) are all 1’s, i.e., = 1 J ×2 K , the corresponding model is a general mixture of Bernoulli products. Theorem 1 implies that a mixture of Bernoulli products is generically identifiable up to label switching, provided that J ≥ 2K + 1, which is consistent with Corollary 5 in Allman et al. (2009). If the ∗ entries in condition (G1) are all 0’s except the intercept, the corresponding Q-matrix is similar to (5), the Q-matrix from Xu (2017) and Xu and Shang (2017) for strict identifiability. In fact, the technique we use to prove generic identifiability (Theorem 1) can be easily extended to show strict identifiability (Theorem 2) under the monotonicity constraints (4) and the following two conditions. YINYIN CHEN ET AL. ⎛ ⎞ D1 (S1) The true sparsity matrix takes the form of = ⎝ D2 ⎠ after row swapping, where K is a (J − 2K ) × 2 binary matrix and D1 = D2 = Ds with ⎡ 1 0 ... 0 ... 0 1 ... 0 ... .. . . . . 1 0 0 ... 1 ... 1 ⎢1 ⎢ Ds = ⎢ . ⎣ .. ⎤ 0 0⎥ ⎥ .. ⎥ . .⎦ 0 (S2) For any two classes of subjects, there exists at least one item in such that they have different positive response probabilities. Theorem 2 (Strict identifiability) Any parameter from (π, B) is strictly identifiable, if conditions (S1) and (S2) and the monotonicity constraints (4) hold. Theorem 2 is similar to the strict identifiability results in Xu (2017) and Xu and Shang (2017), although our proof technique, which is based on tensor product, is different from theirs. The proof of Theorem 2 is given in “Appendix B.2.” 3.5. Proof of Generic Identifiability In this section, we describe the core of our proof for Theorem 1, which is based on the results in Kruskal (1976, 1977) for the uniqueness of tensor decomposition and the tripartition approach in Allman et al. (2009). First, we introduce some notation. Definition 3 The class-response matrix M(, B) is defined as a matrix of size 2 K × 2 J , where the entries are indexed by a row index α and a column index y. The αth row and yth column element of M(, B) is the probability that a subject with attribute profile α gives response y, i.e., P(Y = y|α, B) = J y θ j,αj (1 − θ j,α )1−y j . j=1 Definition 4 For a class-response matrix M, the Kruskal rank of M is the largest number I such that every I rows of M are independent. Remark 4 If M is full row rank, the Kruskal rank of M is its row rank. Next, we reformulate a result in Allman et al. (2009) as follows. Theorem 3 (Allman et al. 2009) Consider a general latent class model with r classes and J features, where J ≥ 3. Suppose all entries of π are positive. If there exists a tripartition of the set J = {1, 2, · · · , J } that divides J into three disjoint, nonempty subsets J1 , J2 , J3 such that the Kruskal ranks of the three class-response matrices M 1 , M 2 , and M 3 satisfy I1 + I2 + I3 ≥ 2r + 2, (6) where Is denote the Kruskal rank of M s for features in Js , then the parameters of the model are uniquely determined, up to label switching. PSYCHOMETRIKA We can view Theorem 3 from the perspective of tensor decomposition. The distribution of Y can be represented as a 2|J1 | × 2|J2 | × 2|J3 | -dimensional three-way tensor T according to the tripartition J1 , J2 , and J3 defined in Theorem 3. The ( y1 , y2 , y3 )th element is the probability of observing ( y1 , y2 , y3 ), namely, T ( y1 , y2 , y3 ) = P(Y J1 = y1 , YJ2 = y2 , Y J3 = y3 |π, B) πα P(Y J1 = y1 , Y J2 = y2 , Y J3 = y3 |B, α) = α = πα P(Y J1 = y1 |B, α)P(Y J2 = y2 |B, α)P(Y J3 = y3 |B, α), α where the last equation is due to the fact that given attribute profile α, components of Y are independent. Therefore, identifiability is equivalent to the uniqueness of the following tensor decomposition: T= πα M 1,α ⊗ M 2,α ⊗ M 3,α α = M̃ 1,α ⊗ M 2,α ⊗ M 3,α , α where M s,α is the αth row of M s , and M̃ 1,α = πα M 1,α . Results from Kruskal (1976, 1977) state that if the sum of the Kruskal ranks of M̃ 1 , M 2 , M 3 is larger than or equal to 2r + 2, then the tensor decomposition is unique up to simultaneous permutation and rescaling of the rows. Since M 1 , M 2 , M 3 are all class-response matrices, of which every row sums to 1, the uniqueness of tensor decomposition implies model identifiability. Now, we are ready to give our proof. We divide into D1 , D2 , , which correspond to J1 , J2 , J3 , respectively. For this tripartition, both J1 and J2 contain K items and their sparsity matrices are from Dg , respectively; the remaining J − 2K items are included in J3 corresponding to satisfying condition (G2). Accordingly, we decompose the parameter space into three parts, (B) = D1 ⊗ D2 ⊗ . To check inequality (6), we show that under conditions (G1) and (G2), we have I1 = 2 K , I2 = 2 K and I3 ≥ 2 hold almost everywhere in D1 , D2 , , respectively. Consequently, identifiability holds almost everywhere in (B). In Theorem 4, we show that for any D ∈ Dg , the class-response matrix M( D, B) with size 2 K × 2 K , is of rank 2 K (full rank) almost everywhere in D . Therefore, with condition (G1), we have I1 + I2 = 2·2 K hold (almost everywhere in D1 ⊗ D2 ). To prove Theorem 1, it then suffices to show that with condition (G2), we have I3 ≥ 2 (almost everywhere in ). In fact, under condition (G2), the following statement holds almost everywhere in : for any two different classes, α 1 , α 2 , there must exist one j0 > 2K , such that θ j0 ,α 1 = θ j0 ,α 2 . The exceptional case is that when β jk ,k = 0 holds for some k, which is of Lebesgue measure zero with respect to . J Note that the αth row of M 3 is given by M 3,α = j=2K +1 (θ j,α , 1 − θ j,α ). So θ j0 ,α 1 = θ j0 ,α 2 implies M 3,α 1 = M 3,α 2 . Therefore, rows of M 3 are unique, which implies that the Kruskal rank of M 3 is at least 2, i.e., I3 ≥ 2, (almost everywhere in ). Theorem 4 Given D ∈ Dg , the corresponding class-response matrix, M( D, B), is of full rank except some values of B from a measure zero set with respect to D , i.e., λ D {B ∈ D : det[M( D, B)] = 0} = 0, where λ D {A} denotes the Lebesgue measure of set A with respect to D . (7) YINYIN CHEN ET AL. The proof of Theorem 4 for general D ∈ Dg is given in “Appendix B.1.” Next, we look at two special cases. Example 1 Da = 1 K ×2 K . Proof Da implies a general mixture of K -dimensional Bernoulli products with 2 K classes. There K exists a one-to-one mapping between the probability matrix θ = {θ j,α } ∈ [0, 1] K ×2 and the K coefficient matrix B ∈ Da (B) = R K ×2 . Equation (7) holds if the solution set det[M( D, B)] = K det[M(θ )] = 0 in terms of θ is of measure zero with respect to [0, 1] K ×2 , i.e., λ[0,1] K ×2 K {θ ∈ [0, 1] K ×2 : det[M(θ)] = 0} = 0. K Note that det[M(θ )] is a polynomial function of θ with finite degrees and is not constantly zero K for any θ ∈ [0, 1] K ×2 , the solution set forms a proper subvariety which must be of dimension K less than K × 2 (Cox et al. 1994; Allman et al. 2009), hence, of Lebesgue measure zero. Example 2. Ds = (1 K , I K , 0). Proof. Ds implies a DINA model with K items, K skills and Q = I K . For any item, β j can be reparameterized by the guessing parameter g j = (β j,0 ) and the slipping parameter s j = 1 − (β j,0 + β j, j ). We can rewrite the class-response matrix as the Kronecker product of K of 2 × 2 sub-matrices, K M( Ds , B) = j=1 gj 1 − gj := 1 − sj sj K Mj j=1 By the property of Kronecker products, we have rank(M( Ds , B)) = Kj=1 rank(M j ). Therefore, M( Ds , B) is of full rank unless there exists at least one j0 , such that g j0 = 1 − s j0 , or equivalently β j0 , j0 = 0. Therefore, the dimension of the exceptional set is less than the dimension of Ds , hence of Lebesgue measure zero. Remark 5. In the above two special cases, after re-parameterizing B, we simply compare the dimension of the solution set and that of the parameter space. This technique, however, cannot be applied to general settings where the equation det[M( D, B)] = 0 is not easy to solve. 4. MCMC for Model Estimation 4.1. Bayesian Formulation Consider a SLCM with N subjects, J items and K skills. Let α i denote the attribute profile of subject i, and Yi j denote the response of subject i to item j. Symbols like β j and δ j are defined in Sect. 2. Throughout, we use subscript i = 1, . . . , N to index subjects, j = 1, . . . , J to index items, and p = 1, . . . , 2 K to index elements of the coefficients with p = 1 corresponding to the intercept. We formulate the Bayesian model as follows. PSYCHOMETRIKA • The probit model on Yi j : Yi j | α i ∼ Bernoulli β Tj aαi , where (·) is the probit link function. • The spike-and-slab prior on B: ! β jp | δ jp ∼ N (0, σβ2 ) δ jp = 1 , I(β j p = 0) δ j p = 0 δ j p |ω ∼ Bernoulli (ω) , ω ∼ Beta(w0 , w1 ), where I is an indicator function. The intercept is always set active with δ j1 = 1. In K addition, the prior distribution on B is restricted on B(B), a subset of R J ×2 where the monotonicity constraints (4) and strict/generic identifiability conditions are satisfied. • The prior on latent attributes α i : α i | π ∼ Multinomial(π ), π ∼ Dirichlet(d 0 ). Here, (σβ2 , w0 , w1 , d 0 ) are user-specified hyper-parameters. A similar sparsity inducing prior specification was used by Culpepper (2019) for general CDMs. 4.2. The Gibbs Sampling Algorithm Following the data augmentation approach in Albert and Chib (1993), we introduce an augmented variable Z i j ∼ N (β Tj aαi , 1) and Yi j = I(Z i j > 0). In particular, let Z j denote the N augmented variables for item j, and Z j can be written as the following Gaussian regression model, Z j α 1:N , β j ∼ N N Xβ j , I N , T where X = aα 1 , . . . , aα N is an N × 2 K design matrix shared for all j = 1, . . . , J. Although vanilla MCMC algorithms are available for Bayesian variable selection with Gaussian regression models, they are not applicable here. We derive a tailored MCMC scheme for our model, which ensures that each posterior draw of B is from the valid parameter space B(B). A key step in our algorithm is to sample the indicator variable δ j p . First, we need to decide whether δ j p is allowed to be changed, i.e., whether the identifiability conditions and the monotonicity constraints still hold when δ j p is replaced by 1 − δ j p . For strict identifiability or generic identifiability, the explicit conditions of are given in Sect. 3. Checking whether a matrix satisfies these conditions is straightforward. If these conditions are satisfied either with δ j p = 0 or 1, then δ j p is allowed to be updated, otherwise leave δ j p unchanged in this iteration. Next, we derive the sampling distribution for δ j p if it is allowed to be updated. Since δ j p is deterministic if β j p is given, we need to derive the conditional distribution of δ j p given other variables except β j p , namely, δ j p Z j , α 1:N , β j ( p) , ω, σβ2 ∼ Bernoulli ω̃ j p , (8) YINYIN CHEN ET AL. where β j ( p) is the coefficient vector β j without the pth element and ω̃ j p is the Bernoulli parameter we need to determine. To compute ω̃ j p , we need to evaluate the integrated likelihood of Z j with respect to β j p , as the evidence for δ j p = 1. Express the density function of Z j as T N 1 " Z j − X pβ jp Z j − X pβ jp p Z j α 1:N , β j = (2π )− 2 exp − " 2 (9) where " Z j = Z − X ( p) β j ( p) , and X ( p) is an N × 2 K − 1 matrix that excludes the pth column. If there is no constraint on β j p , it is straightforward to compute the integrated likelihood of (9) with respect to β j p ∼ N (0, σβ2 ). However, with the monotonicity constraints on B, it is not clear what kind of values β j p is allowed to take. Recall the monotonicity constraints on B: for j = 1, . . . , J , min β Tj aα = β j1 , α max β Tj aα < min β Tj aα . α:aα δ j α:aα δ j Next, we show that if β j p is lower bounded, the constraints above are satisfied. Proposition 1. Suppose the coefficient matrix in iteration t satisfies identifiability conditions and the monotonicity constraints, B (t) ∈ B(B), and only β j p ( p > 1) is sampled in iteration t + 1. If (t+1) β j p > L, then B (t+1) ∈ B(B). The lower bound L is given by # L = max max (−γ j,α ), max γ j,α − γ α∈L1 (t)T (t) α∈L0 $ (t) j,q j , (t) = β where γ j,α aα − β j1 − β j p , L1 = {α|aα, p = 1}, L0 = {α|aα, p = 0, α 0} and aα, p j denotes the pth element of aα . The proof of Proposition 1 is given in “Appendix E.” Using Proposition 1, we can show that the Bernoulli parameter in (8) is given by ω̃ j p = −L σβ −L σβ −1 ω −1 ω σ̃ p σβ σ̃ p σβ μ̃ j p −L σ̃ p μ̃ j p −L σ̃ p exp # 2 1 μ̃ j p 2 σ̃ p2 exp # 2 $ 1 μ̃ j p 2 σ̃ p2 $ , +1−ω −1 where X p denotes the pth column of matrix X, σ̃ p2 = X p X p + σβ−2 and μ̃ j p = −1 . See “Appendix D” for the detailed derivation. X p " Z j X p X p + σβ−2 After updating δ j p , we update β j p based on the full conditional distribution below: % &δ % &1−δ j p β j p Z j , α 1:N , β j ( p) , σβ2 , δ j p ∼ N μ̃ j p , σ̃ p2 I β j p > L j p I β j p = 0 . (10) We summarize the sampling algorithm of B and in Algorithm 1, and the full sampling algorithm in Algorithm 2. PSYCHOMETRIKA Algorithm 1: Sample B and for j ← 1 to J do for p ← 1 to 2 K do K for α ∈ {0, 1}2 do γα ← β Tj aα − β j0 − β j p end −1 −1 Z j X p X p + σβ−2 σ̃ p2 ← X p X p + σβ−2 ," Z j = Z − X ( p) β j ( p) , μ̃ j p ← X p " . L ← max{maxα∈L1 −γα , maxα∈L0 γα − γq }, j if ( L ≤ 0) and (δ j p = 1 − δ j p satisfies the identifiability conditions) then ⎞1 ⎞ ⎛ ⎛ # $ μ̃2j p μ̃ j p −L σ̃ p2 2 exp⎝ 21 2 ⎠ ω⎝ 2 ⎠ σ̃ p σβ σ̃ p . ⎞ ⎛ ⎛ ⎞1 $−1 # # $ 2 μ̃ μ̃ j p −L σ̃ 2 2 −L ⎝ p⎠ ⎝ 1 j p ⎠+1−ω ω exp σβ 2 σ̃ 2 σ̃ p σβ2 p # ω̃ j p ← −L σβ $−1 Sample δ j p from Bernoulli (ω̃ j p ). end if δ j p = 1 then Sample β j p from truncated normal distribution, N (μ̃ j p , σ̃ p2 )I (β j p > L). end if δ j p = 0 then β j p = 0. end end end return B , 4.3. Bayesian Penalization In this subsection, we show that the spike-and-slab prior, a Bayesian variable selection technique, is equivalent to a mixture of L 0 and L 2 penalty terms on B. (1 − ω)2 , and the prior distribution of π 2π ω2 is Dirichlet(1), then the Bayesian MAP estimate of (π , B) is equivalent to the optimum of the objective function with a mixture of L 0 and L 2 penalty terms on B, i.e., Proposition 2. If σβ2 , ω are fixed such that σβ2 > ' ( arg max p(π , B|Y N ) = arg max − (π , B|Y N ) + λ1 B0 + λ2 B2 π ,B π,B where Y N is the observed responses matrix of N test subjects, (π, B|Y N ) is the log-likelihood function, and λ1 , λ2 > 0 are constants. Proof. The marginal posterior distribution of (π, B) is written as p(π , B|Y N ) ∝ f (Y N |π, B) p(π) p(B|σβ2 , ) p(|ω) YINYIN CHEN ET AL. Algorithm 2: Full Gibbs sampling algorithm Input: Y N ×J , initial values of π , α 1:N , B , total chain length T , burn-in period b, and hyper-parameters σβ2 , w0 , w1 , d 0 . for t ← 1 to T do for j ← 1 to J do for α ∈ {0, 1} K do θ j,α ← (aT α β j ). end end for i ← 1 to N do Sample α i from the multinomial distribution, yi j 1−yi j P(α i = α|π , yi ) ∝ πα Jj=1 θ j,α (1 − θ j,α ) end for α ∈ {0, 1} K do )N Ñα ← i=1 1(α i = α) Sample π from Dirichlet2 K ( Ñ + d 0 ) end for j ← 1 to J do Sample Z j from the truncated normal distribution, N y 1−yi j N N (Xβ j , I N ) i=1 [I (Z i j > 0)] i j [I (Z i j < 0)] . end Sample B and using )Algorithm 1, ) Sample ω from Beta j, p (1 − δ j p ) + ω0 , j, p δ j p + ω1 . if t > b then B (t−b) ← B , (t−b) ← end end return B (1) · · · , B (T −b) , (1) · · · , (T −b) . Taking natural logarithm on both sides and ignoring the constant, we have log p(B, π |Y N ) =(Y N |π, B) + log p(π) + log p(B|σβ2 , ) + log p(|ω) * 1 ω B2 − B0 log =(π , B|Y N ) − B0 log 2π σβ2 − 2 1 − ω 2σβ * ω 2π σβ2 1 − =(π , B|Y N ) − B0 log B2 1−ω 2σβ2 =(π , B|Y N ) − λ1 B0 − λ2 B2 , where λ1 = log * ω 2π σβ2 1−ω and λ2 = 1 . 2σβ2 Remark 6. Although L 0 penalty is the natural choice to account for the model complexity, it is computationally inefficient. For tractable computation, Xu and Shang (2017) proposed to use truncated lasso penalty (TLP) as an approximation, while we use the spike-and-slab prior. 5. Monte Carlo Simulation 5.1. Overview To test the performance of the proposed algorithm, we employ Monte Carlo simulation, using different attribute sizes (K = 3, 4), different sample sizes (N = 500, 1000, 2000), and different PSYCHOMETRIKA correlations among attributes (ρ = 0, 0.15, 0.25). The unknown true Q-matrices, with J = 20 items, satisfying the strict identifiability conditions, are as follows: ⎞ ⎛ ⎞ ⎛ 1000 100 ⎜0 1 0 0⎟ ⎜0 1 0⎟ ⎟ ⎜ ⎟ ⎜ ⎜0 0 1 0⎟ ⎜0 0 1⎟ ⎟ ⎜ ⎟ ⎜ ⎜0 0 0 1⎟ ⎜1 0 0⎟ ⎜ ⎟ ⎜ ⎟ ⎜1 0 0 0⎟ ⎜0 1 0⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 1 0 0⎟ ⎜0 0 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 0 1 0⎟ ⎜1 0 0⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 0 0 1⎟ ⎜0 1 0⎟ ⎜ ⎟ ⎜ ⎟ ⎜1 1 0 0⎟ ⎜0 0 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 1 1 0⎟ ⎜1 1 0⎟ ⎜ ⎟ ⎟ Q = Q=⎜ ⎜0 0 1 1⎟. ⎜1 0 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜1 0 0 1⎟ ⎜0 1 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜1 0 1 0⎟ ⎜1 1 0⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 1 0 1⎟ ⎜1 0 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜1 1 0 0⎟ ⎜0 1 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 0 1 1⎟ ⎜1 1 0⎟ ⎜ ⎟ ⎜ ⎟ ⎜0 1 1 1⎟ ⎜1 0 1⎟ ⎜ ⎟ ⎜ ⎟ ⎜1 0 1 1⎟ ⎜1 1 1⎟ ⎜ ⎟ ⎜ ⎟ ⎝1 1 0 1⎠ ⎝1 1 1⎠ 1110 111 For the ρ = 0 cases, the attribute profile α is generated uniformly from the 2 K classes. In other words, different attributes are independent. For the ρ > 0 cases, dependence among attributes is introduced using the method of Chiu et al. (2009). Specifically, ξ = (ξ1 , . . . , ξ K ) is generated from the multivariate normal distribution N (0, ) with variance 1 and correlation ρ such that = (1 − ρ)I K + ρ1 K ×K . The attribute profile α = (α1 , . . . , α K ) is given by αk = I(ξk ≥ 0), k = 1, . . . , K . The positive response probability of each item is set between 0.2 and 0.8 using the method of Xu and Shang (2017). In particular, we set the probability of attribute profiles with K j out of the K j relevant attributes to be 0.2 + (0.8 − 0.2) × K j /K j . Our model does not explicitly include a Q-matrix, but in order to compare with other methods, we recover the Q-matrix by aggregating B̂, the posterior mean of B, in the following way: 1. sum up the values of the relevant coefficients (t) (t) q̃ jk = β̂ j,k + (t) β̂ j,k1 k + k1 (t) β̂ j,k1 k2 k + . . . , t = 1, . . . , T k1<k2 2. standardize )T q̄ jk = (t) t=1 q̃ jk )T (t) maxk∈{1,...,K } t=1 q̃ jk 3. identify q̂ jk as 1 if q̄ jk exceeds a fixed threshold. Here, we set the cutoff to 0.5 in all configurations q̂ jk = 1{q̄ jk > cutoff} . YINYIN CHEN ET AL. Table 3. Recovery of Q for K = 3 ρ 0 0.15 0.25 N 500 1000 2000 500 1000 2000 500 1000 2000 Strictly identifiable space Generically identifiable space Matrix Item TPR FPR Matrix Item TPR FPR 0.582 0.936 1.000 0.530 0.934 0.998 0.540 0.960 0.994 0.974 0.997 1.000 0.970 0.997 1.000 0.971 0.998 1.000 0.988 0.999 1.000 0.988 0.999 1.000 0.989 0.999 1.000 0.005 0.001 0.000 0.008 0.001 0.000 0.009 0.001 0.000 0.586 0.946 0.998 0.564 0.928 0.996 0.508 0.898 0.990 0.975 0.997 0.999 0.971 0.996 1.000 0.966 0.995 1.000 0.989 0.999 1.000 0.990 0.999 1.000 0.990 0.999 1.000 0.006 0.001 0.000 0.010 0.001 0.000 0.014 0.003 0.000 Table 4. Recovery of B for K = 3 and K = 4 ρ N Strictly identifiable space K =3 0 0.15 0.25 1000 2000 1000 2000 1000 2000 Generically identifiable space K =4 K =3 K =4 RMSE aBias RMSE aBias RMSE aBias RMSE aBias 0.112 0.074 0.108 0.075 0.106 0.077 0.065 0.044 0.064 0.045 0.064 0.047 0.114 0.091 0.108 0.091 0.107 0.095 0.057 0.047 0.057 0.049 0.058 0.052 0.115 0.075 0.109 0.077 0.110 0.080 0.071 0.046 0.069 0.049 0.071 0.052 0.128 0.088 0.136 0.112 0.145 0.129 0.086 0.059 0.096 0.074 0.102 0.080 5.2. Results We generated 500 independent replications for each configuration. Table 3 summarizes the results of the recovery of the Q-matrix for K = 3. The metrics we use here are the same as in Xu and Shang (2017). The column “Matrix” records the matrix-level Q-matrix recovery rates of the 500 replications. The column “Item” gives the item-level recovery rates. “TPR” and “FPR” are two entry-level rates. The column “TPR” is the true positive rate, i.e., the proportion of 1’s in the true Q-matrix correctly estimated; “FPR” is the false positive rate, i.e., the proportion of 0’s in the true Q-matrix incorrectly estimated as 1’s. Table 4 summarizes the results of the recovery of B for K = 3 and K = 4. RMSE is the averaged root-mean-square error, and aBias is the averaged absolute values of the estimated biases. For each configuration, we restricted the parameter search to the strictly identifiable space (under strict identifiability conditions) and the generically identifiable space (under generic identifiability conditions), respectively; we report the results in the columns “Strictly Identifiable Space” and “Generically Identifiable Space.” The recovery rates of the former are generally higher. Note that the true parameters live in both spaces, and the strictly identifiable space is much smaller than the generic one. The posterior draws might be likely to be closer to the true parameters if we were to restrict the parameter search to a smaller space. However, we observe that such differences of accuracy become smaller when the sample size becomes larger. PSYCHOMETRIKA Table 5. Computation time (minutes) per replication K =3 K =4 N = 500 N = 1000 N = 2000 2.94 3.82 4.91 6.23 8.48 11.38 It is worth mentioning that our MCMC algorithm is quite efficient. It can provide estimates in several minutes. Table 5 gives average computation times for a Markov chain with 30,000 burn-in samples and 10,000 post-burn-in samples in a MacBook Pro (Retina, Early 2015) with 2.9 GHz Intel Core i5 processor. 6. Real Data Analysis In this section, we apply our algorithm to two real applications. The first dataset is Tatsuoka’s Fraction Subtraction dataset (Tatsuoka 1984, 2002). The second dataset is from an experimental IQ test offered on the Open Psychometrics website.1 6.1. Fraction Subtraction Data This dataset contains responses to a set of fraction subtraction items collected from N = 536 middle school students. It has been widely analyzed in the literature, e.g., Xu and Shang (2017), Chen et al. (2018), Chen et al. (2015), de la Torre and Douglas (2004). Table 6 presents the estimated coefficient matrix by our Gibbs sampling algorithm with K set at 3. The estimated B is one of the posterior draws in the Markov chain such that its corresponding matrix is closest to the average of among all the posterior draws in the chain. It is not hard to see that the estimated B follows the generic identifiability conditions and the monotonicity constraints. The three attributes can be roughly interpreted as (i) applying subtraction to the integer and the fraction separately, (ii) determining common denominator, (iii) converting to improper fraction. The detected skills are consistent with the findings of Chen et al. (2015, 2018). The estimated coefficients illustrate one benefit of focusing on versus Q. That is, it provides insight into how underlying attributes function together. For example, Item 4, 3 21 − 2 23 , requires improper fraction conversion, followed by common denominator determination and subtraction. In our results, Item 4 has three positive coefficients, β3 , β13 and β23 . It suggests that mastering improper fraction conversion, skill (iii), helps to solve the item. Conditioning on the mastery of skill (iii), knowing skill (i) and (ii) increases the success rate. But for those who do not master skill (iii), mastery of the other two skills does not compensate. In most previous analyses (e.g., Xu and Shang 2017; Chen et al. 2015, 2018), the estimated Q-matrices suggest only the crucial requirement of skill (iii); they do not indicate that skills (i) and (ii) are also relevant. Another observation is that the signs of interaction terms imply the logic gates of the involved skills. For example, the estimated β13 ’s of Items 5, 6, and 7 are positive, while those of Items 9 and 10 are negative. In fact, the former three items all have two distinct solution paths: (a) convert the first number to an improper fraction (skill (iii)), and then apply subtraction; or (b) apply subtraction to the integer and the fraction separately (skill (i)). The negative β13 ’s of those items indicate that 1 https://openpsychometrics.org/_rawdata/. YINYIN CHEN ET AL. Table 6. Estimated B for fraction subtraction data Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Content 5 − 3 3 4 3 3 4 − 8 5 − 1 6 9 3 21 − 2 23 1 18 − 18 3 45 − 3 25 4 57 − 1 47 4 4 35 − 3 10 3 − 2 15 2 − 13 4 −2 7 4 12 12 4 13 − 2 43 7 35 − 45 1 −2 8 4 10 10 4 − 1 43 4 13 − 2 53 3 38 − 2 56 β0 β1 β2 β3 β12 − 1.74 2.80 − 1.54 2.18 1.46 − 2.65 3.72 0.03 − 0.81 β13 0.29 1.12 0.42 1.56 0.55 2.24 − 0.97 − 1.21 2.06 0.72 2.27 − 1.55 − 1.44 1.99 0.65 1.89 − 1.01 − 0.68 0.09 0.42 − 2.48 0.70 1.78 0.83 1.81 − 2.24 1.02 1.66 1.30 1.49 0.85 1.94 0.64 − 1.55 2.19 − 2.12 3.06 − 1.82 0.65 − 3.11 0.65 1.48 − 2.39 − 2.34 0.66 0.89 0.15 0.71 0.66 0.80 2.11 1.97 − 0.59 1.24 2.09 2.90 0.18 0.40 β123 1.23 − 0.99 − 2.14 β23 0.43 0.82 1.15 2.08 − 0.09 students who have mastered either skill (i) or (iii) will have high success probabilities; mastery of both is not expected to increase success rates, implying an “or” gate. In contrast, Items 9 and 10 can be solved only by a two-step solution path, i.e., improper fraction conversion (skill(iii)) followed by separate subtractions (skill(i)). The positive β13 ’s of these items indicate that mastering the two skills together would largely increase the success probability, which implies that these two skills function through an “and” gate. 6.2. Experimental Matrix Reasoning Data This dataset contains responses of N = 400 subjects to a set of IQ test questions. In each question, a matrix with one tile missing is given, as well as eight options for that missing tile. Participants are required to choose the most appropriate tile among the eight options. An example question is given in Fig. 1. We study the J = 20 questions for which the corresponding correct rates are larger than 25%. We list the question matrices in Fig. 2. For more details (the options and the correct answers), please refer to the documentation on Open Psychometrics. We observe that the patterns of Q1, Q2, Q6, Q8, Q10, Q11, Q14-Q20, and Q23-Q25 are row-wise; as a result, for each question, for the participants to figure out the correct answer to each such question, they must learn the pattern from the first two rows and apply it to the last row. For example, in Q2, the three tiles in each complete row are the same, so the last tile in the incomplete row must be the same as the ones to its left. In Q6, the tiles from left to right on each row are sequentially rotated 90 degrees in the clockwise direction, so we recover the missing tile by 90 degrees clockwise rotation of the tile to its left. On the other hand, for Q1, Q3, Q4, Q5, Q12, PSYCHOMETRIKA Figure 1. An example question (Q1): the left is the matrix missing one tile, and the right is the eight possible tiles for participants to choose, and the red marked one is the correct answer and Q13, the participants are required to infer the whole picture of the matrix from the provided tiles and find the one tile that can complete the best overall pattern. For example, in Q3, if the participants can recognize that the whole picture contains one small diamond hollow square and one large spade hollow square, then they are able to choose the proper tile, a tile with a diamond on top left corner and spades on the bottom and the right, to perfectly complete a symmetrical pattern. Similarly, the matrix of Q4 is shaped like a number “2”, so we can infer that the last tile should include a horizontal line in the middle. At the same time, for Q6, Q8, Q10, Q11, Q14, Q15, Q19, Q20, Q23, and Q24, the missing tiles are similar to some tiles on the same matrices but with little change, including suites changes (Q19, Q20, Q23), rotation (Q6, Q14), and stretching (Q8). However, for Q1, Q2, Q4, Q5, Q9, Q16, and Q25, the missing tiles are exact the same as one of their neighboring tiles. From the above analysis, we summarize the four skills to solve these questions as (i) learning the row-wise pattern, (ii) learning the pattern from the whole picture, (iii) changing from neighbors, and (iv) copying from neighbors. Table 7 presents the estimated coefficients provided by our algorithm with K set at 4; the four estimated attributes are roughly consistent with the four skills analyzed above. 7. Discussion This work focuses primarily on the study of model identifiability and parameter estimation of SLCM, in which most of the challenges are caused by the restrictions introduced from the context of CDMs. We prove the generic identifiability conditions for SLCM, which relaxes the constraints in the strict identifiability conditions in Xu (2017) and Xu and Shang (2017). We develop a Gibbs sampling algorithm for SLCM, which enforces identifiability conditions and the monotonicity constraints for valid posterior inference. The simulation results demonstrate that our algorithm YINYIN CHEN ET AL. Figure 2. Twenty questions of IQ test PSYCHOMETRIKA Table 7. Estimated B for IQ test Data: the boldfaced coefficients are the ones consistent with the analysis β0 Q1 Q2 Q3 Q4 Q5 Q6 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q19 Q20 Q23 Q24 Q25 0.0 0.3 0.0 − 0.6 0.1 − 0.2 − 0.6 0.0 − 0.7 − 0.2 − 1.0 − 0.9 − 0.8 − 0.8 − 0.2 − 0.8 − 1.0 − 1.1 − 0.9 − 0.3 β1 β2 β3 β4 1.3 1.0 0.7 0.7 1.6 1.1 1.0 0.8 0.5 0.9 0.3 0.5 1.3 0.4 0.6 1.6 0.9 0.7 1.6 0.9 1.5 0.9 0.4 0.8 2.0 1.6 0.7 1.2 1.0 0.2 1.0 β12 β13 β14 β23 β24 β34 β123 β124 β134 β234 β1234 0.3 0.4 0.5 0.9 0.6 0.5 1.1 0.2 1.2 0.9 0.2 0.7 0.4 0.2 0.5 0.4 0.1 efficiently estimates the model in different configurations, with accuracy comparable to that of alternative models. Our proof for generic identifiability is based upon the technique of generic identifiability in Allman et al. (2009) and the sufficient conditions of Kruskal (1976, 1977) for the uniqueness of three-way tensor decomposition. We note that the conditions we provide are sufficient but may not be necessary. We note further that Xu (2017) has developed a different technique, working directly on the class-response matrix. An interesting direction for future research is to draw connections between the two sets of techniques, perhaps shedding light on the identification of sufficient and necessary conditions for model identifiability for SLCM. In this study, the number of attributes, K , is assumed to be known and fixed, a limitation in practice, because it is usually difficult to specify K beforehand, especially when no prior information is available. It is always of interest to study the model with unknown K , and to develop algorithms to select K in model fitting. One promising method is nonparametric Bayesian, which is able to infer from the data an adequate model size. Nevertheless, the model identifiability may continue to present challenges. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. A Connections of SLCM to Popular CDMs In this section, we discuss the connections between SLCM and popular CDMs. To simplify the expression, we assume the relevant skills of item j are k1 , . . . , k R , i.e., q jk1 = · · · = q jk R = 1, q jk = 0, otherwise. YINYIN CHEN ET AL. Example 3. (DINA model). The deterministic input noisy output “and” gate model (Haertel 1989; Junker and Sijtsma 2001) is a conjunctive model. It assumes that a student is most capable of answering question j positively only if he/she masters all of its relevant skills. The item response function takes the following form, P(Y j = 1|α, q j ) = (1 − s j )1(α q j) g j 1(αq j ) , where s j = P(Y j = 1|α q j ) is the slipping parameter, which is the probability that a student capable for item j but response negatively and g j = P(Y j = 1|α q j ) is the guessing parameter, which is the probability that a non-master answers positively. It is assumed that g j < 1 − s j in most applications. The DINA model can be written as P(Y j = 1|α, β j ) = β j,0 + β j,k1 ...k R αk1 . . . αk R where only one coefficient, besides the intercept, in β j is active, δ j,0 = δ j,k1 ...k R = 1, δ j, p = 0 otherwise. The guessing parameter g j and slipping parameter s j is given by, g j = (β j,0 ), s j = 1 − (β j,0 + β j,k1 ...k R ). Example 4. (DINO model). The deterministic input noisy output “or” gate model (Templin and Henson 2006) is a disjunctive model, which assumes that a student is capable to answer question j positively if at least one of the relevant skills is mastered. The item response function is P(Y j = 1|α, q j ) = (1 − s j )1(α Tq j >0) g j 1(α Tq j =0) where s j and g j are defined the same as in DINA, and g j < 1 − s j is assumed. The DINO model can be reparameterized as ⎛ P(Y j = 1|α, β j ) = ⎝β j,0 + R r =1 β j,kr αkr + kr >kr β j,kr kr αkr αkr + · · · + β j,k1 ...k R R ⎞ αkr ⎠ r =1 where the coefficients containing only the relevant skills are active, δ j,0 = δ j,k1 = · · · = δ j,k R = δ j,k1 k2 = · · · = δ j,k R−1 k R = · · · = δ j,k1 ...k R = 1, δ j, p = 0 otherwise The coefficients with odd orders are all equal and positive. The coefficients with even orders are the additive inverse of those with odd orders. β j,k1 = β j,k2 = · · · = β j,k R = β j,k1 k2 k3 = · · · = β j,k R−2 k R−1 k R = · · · = − β j,k1 k2 = · · · = −β j,k R−1 k R = −β j,k1 k2 k3 k4 = · · · = −β j,k R−3 k R−2 k R−1 k R = · · · . PSYCHOMETRIKA The guessing parameter g j is in the same form of the one in DINA model and slipping parameter s j is given by 1 − (aTα β j ), with α satisfying α T q j > 0, which is equivalent to 1 − (β j,0 + β j,kr ), r = 1, . . . , R, g j = (β j,0 ), s j = 1 − (β j,0 + β j,kr ), r = 1, . . . , R. Example 5. (G-DINA model). The DINA model is generalized to the G-DINA model by de la Torre (2011), which takes the form of P(Y j = 1|α, q j ) = β j,0 + K β j,k q jk αk + k>k k=1 K β j,kk q jk αk q jk αk + · · · + β j,12...K q jk αk . k=1 By using the identity link in Eq. (1), it can be written as P(Y j = 1|α, β j ) = β j,0 + R β j,kr αkr + R β j,kr kr αkr αkr + · · · + β j,k1 ...k R αkr kr >kr r =1 r =1 where the coefficients containing only the relevant skills are active, δ j,0 = δ j,k1 = · · · = δ j,k R = δ j,k1 k2 = · · · = δ j,k R−1 k R = · · · = δ j,k1 ...k R = 1, δ j, p = 0 otherwise. Example 6. (NC-RUM model). Under the reduced noncompensatory reparameterized unified model (DiBello et al. 1995; Rupp et al., 2010), attributes have a noncompensatory relationship with observed response. It assumes missing any relevant skill would inflict a penalty on the positive response probability. P(Y j = 1|α, q j ) = b j K q (1−αk ) r j,kjk k=1 b j is the positive response probability for students who possess all relevant skills and r j,k , 0 < r j,k < 1, is the penalty for not mastering kth attribute. As pointed by Xu (2017), by using the exponential link function, NC-RUM can be equivalently written as P(Y j = 1|α, β j ) = exp β j,0 + R β j,kr αkr , r =1 where the main effects of relevant attributes are active, δ j,0 = δ j,k1 = · · · δ j,k R = 1, δ j, p = 0 otherwise. The parameters are given by b j = exp β j,0 + R r =1 ! β j,kr , r j,k = exp(−β j,kr ), if k ∈ {k1 , . . . , k R } 1, otherwise YINYIN CHEN ET AL. Example 7. (C-RUM model). Compensatory-RUM (Hagenaars 1993; Maris 1999) is given by, )K exp β j,0 + k=1 β j,k q jk αk . P(Y j = 1|α, q j ) = )K exp β j0 + k=1 β j,k q jk αk + 1 Equivalently, P(Y j = 1|α, q j ) = logit −1 β j,0 + R β j,kr αkr r =1 where (·) is the inverse of the logit function and the main effects of relevant attributes are active, δ j,0 = δ j,k1 = · · · δ j,k R = 1, δ j, p = 0 otherwise. B Proof of Theorems In this section, we provide the proof of Theorems 4 and 2. B.1 Proof of Theorem 4 We first introduce Lemma 5 (Mityagin 2015; Dang 2015) which shows that the solution set of a real analytic function is of Lebesgue measure zero if the function is not constantly 0. Then in Proposition 3, we show that G D (B) := det[M( D, B)] is a real analytic function, and in Proposition 4, we show that G D (B) is not constantly zero for any B ∈ D (B) if D ∈ Dg , so that Lemma 5 applies and Theorem 4 is proved. Lemma 5. (Mityagin 2015; Dang 2015) If f : Rn → R is a real analytic function which is not identically zero, then the set {x : f (x) = 0} has Lebesgue measure zero. Proposition 3. G D (B) = det[M( D, B)] : D → R is a real analytic function of B. Proof. G D (B) is a composition function: G D (B) = det[M] = h(θ α 0 , . . . , θ α 2 K −1 ) = h (B D aα 0 ), . . . , (B D aα 2 K −1 ) where h(θ ) : [0, 1] K ×2 → R is a polynomial function and (·) is a CDF. (·) is a real analytic function because a CDF is an integral of a real analytic function, and h(θ) is also a real analytic function since it is a polynomial. Therefore, the composition function G D (B) is a real analytic function due to the fact that the composition of real analytic functions is a real analytic function. K Proposition 4. If D ∈ Dg , there exists some B ∈ D (B), s.t., G D (B) = 0. Proof. Let B 1 = (1 K , I K , 0) ∈ D (B), ∀ D ∈ Dg . As shown in Example 2, M( D, B 1 ) is of full rank, so that G D (B 1 ) = 0. Remark 7. G D (B) ≡ 0 is not a trivial conclusion holds for all kinds of D. D ∈ D is a sufficient condition for G D (B) ≡ 0. If D ∈ D, it is possible G D (B) ≡ 0. See the following example. PSYCHOMETRIKA Example 8. Assume K = 3 and the main effect of the first skill is inactive for all items, i.e., δ j,1 = 0, ∀1 ≤ j ≤ K , then D3×8 takes the form ⎡ ∗ ⎣∗ ∗ ∗ 1 ∗ 0 0 0 ∗ ∗ 1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ⎤ ∗ ∗⎦ . ∗ For any B ∈ D (B) and any response y ∈ {0, 1}3 , M α=(0,0,0), y ( D, B) = M α=(1,0,0), y ( D, B). So the two rows of M( D, B) are identical, and M( D, B) is not full row rank, i.e., det[M( D, B)] ≡ 0. By Lemma 5 and Propositions 3 and 4, Theorem 4 is proved. B.2 Proof of Theorem 2 Proof. As shown in Example 2, for any B ∈ Ds (B), the corresponding class-response matrix is of full rank, rank(M( Ds , B)) = 2 K , holds if and only if, for each item, the success probabilities for students with the relevant skill and those without the relevant skill are different. In fact, if the two probabilities are the same, the monotonicity constraints would be violated. Then, using notation from Sect. 3.5, we conclude that under condition (S1) and the monotonicity constraints, rank(M 1 ) = rank(M 2 ) = 2 K . For M 3 , as each element is nonnegative and each row sums to 1. Under condition (S2), there must exist one item j, such that θ j,α s = θ j,α t , so rank(M 3 ) ≥ 2. C Initialization from the Identifiable Space (0) Initialization of the sparsity matrix J ×2 K : 1. activate the intercepts. (0) Fix the entries in the first column of (0) (i.e., ·1 ) as 1. Denote the remaining J × ˜ (0) . (2 K − 1) sub-matrix as (0) (0) 2. construct D1 and D2 . ˜ (0) to be Fix the first 2K rows of # $ IK 0 . IK 0 ˜ (0) . 3. construct (a) Randomly select K indexes, j1 , . . . , j K , from the set {2K + 1, . . . , J } with ˜ (0) replacement and set jk ,k = 1. YINYIN CHEN ET AL. ˜ (0) by (b) Sample the remaining entries in (0) δ j p |w (0) ∼ Bernoulli(w (0) ), K j > 2K , ( j, p) ∈ / {( jk , k)}k=1 where w (0) ∼ Beta(w0 , w1 ) and w0 , w1 are the parameters of the prior distribution and are treated as fixed. (c) Check the row sum. If any row of (0) sums to 0, then we randomly pick up an entry on this row and set it at 1. 4. shuffle the rows. Draw a J × J permutation matrix P = (e j1 , . . . , e j J ) where ( j1 , . . . , j J ) is a permu˜ (0) ← P ˜ (0) . tation of (1, . . . , J ), and let The above initialization is designed for strict identifiability conditions. To generate a under the generic identifiability conditions, we just need to enlarge the range of entries sampling from the prior distribution in step 3b. Specifically, we change step 3b to ˜ (0) by Sample the remaining entries in (0) δ j p |w (0) ∼ Bernoulli(w (0) ), K ( j, p) ∈ / {( jk , k), (k, k), (K + k, k)}k=1 . (0) Initialization of the coefficients matrix β J ×2 K = (β 1 , . . . , β J )T : (0) (0) β j p = 0, if δ j p = 0, (0) (0) (0) β j p |δ j p = 1 ∝ N (0, σβ2 )I (β j p > 0). D Derivation of ω̃ j p δ j p Z j , α, β j ( p) , ω, σβ2 ∼ Bernoulli ω̃ j p ω̃ j p = ω -∞ L -∞ ω L p Z j α, β j p β j p σβ2 dβ j p p Z j α, β j p β j p σβ2 dβ j p + (1 − ω) p Z j α, β j ( p) , β j p = 0 PSYCHOMETRIKA The numerator is . ω p Z j α, β j p β j p σβ2 dβ j p L . ∞ 1 " − N2 " =ω exp − Z j − A p β j p Z j − A p β j p (2π ) 2 L $−1 # 2 β 1 1 −L j p exp − 2 dβ j p · (2π )− 2 σβ σβ 2σβ # $−1 N −L ω = (2π )− 2 σβ σβ ! 01 / . ∞ 1 1 " − 21 2 " " × dβ j p Ap Ap + 2 β jp − 2 Ap Z j β jp + Z j Z j (2π ) exp − 2 σβ L $ # 2 N −L −1 ωσ̃ p 1 " 1 = (2π )− 2 exp − " Zj Z j Z j + σ̃ p2 Ap " σβ σβ 2 2 / 0 . ∞ 2 1 − 21 1 2 " exp − 2 β j p − σ̃ p A p Z j dβ j p × (2π ) σ̃ p 2σ̃ p L . $ # 2 ∞ 1 −L −1 ωσ̃ p 1 " " 1 μ̃ j p − N2 = (2π ) exp − Z j Z j + (2π )− 2 μ̃ j p −L 2 σβ σβ 2 2 σ̃ p − σ̃ p / $2 0 # $ # − μ̃ − μ̃ β β 1 jp jp jp jp × exp − d 2 σ̃ p σ̃ p # $ # $ $ # 2 μ̃ j p − L 1 μ̃ j p −L −1 ωσ̃ p 1 " " − N2 = (2π ) exp − Z j Z j exp 2 σβ σβ 2 σ p2 σ̃ p ∞ where σ̃ p2 = Ap A p + σβ−2 ω̃ j p = = −1 and μ̃ j p = Ap " Z j Ap A p + σβ−2 N Zj" Zj (2π)− 2 exp − 21 " −L σβ −1 ωσ̃ # 2 $ 1 μ̃ j p exp σβ 2 σ p2 # 2 $ −1 σ̃ μ̃ j p −L p −L 1 μ̃ j p ω exp σβ σβ 2 σ̃ p2 σ̃ p # 2 $ −1 σ̃ μ̃ j p −L p −L 1 μ̃ j p ω exp +1−ω σβ σβ 2 σ̃ 2 σ̃ p N Zj" Zj (2π)− 2 exp − 21 " −L σβ −1 ωσ̃ p # p σβ exp μ̃ j p −L σ̃ p −1 2 1 μ̃ j p 2 σ p2 $ . Accordingly, ω̃ j p is, μ̃ j p −L σ̃ p N + (1 − ω) (2π)− 2 exp − 21 " Zj" Zj p E Proof of Lower Bound L In this section, we derive the lower bound L of β j p (Proposition 1). Suppose at time t, we have a B (t) ∈ B(B) satisfying the monotonocity constraints (4), and we only sample β j p at time t + 1 and leave any other coefficient the same as the one at time t, i.e., (t+1) (t) β js = β js , ∀s = p. YINYIN CHEN ET AL. In what follows, we introduce some notations. We denote β j p and δ j p as β p and δ p , respectively. That is, we omit the subscript of item, j, as the lower bound of coefficient β j p does not depend on any other coefficient of other items. Let γα = β T aα − β0 = −1 (θα ) −!β0 be the sum of the linear component excluding the intercept p γα − β p α ∈ L1 for class α. Further, let γα,− p = p denote the sum of the linear component γα α ∈ L0 , p excluding the intercept and the pth coefficients for class α, where L1 = {α|aα, p = 1} and p L0 = {α|aα, p = 0, α α 0 }. We rewrite the monotonicity constraints (4) as follows, min θα ≥ θα 0 αα 0 max θα < min θα = max θα α∈S0 α∈S1 () α∈S1 where S0 = {α|α q, α α 0 = 0} is the set the classes that not mastering all the relevant skills, and S1 = {α|α q} is the set of classes mastering all the relevant skills. Note that (·) is a strictly increasing function, we have the following equivalent form of the monotonicity constraints (): min γα ≥ γα 0 = 0, αα 0 γq = max γα = min γα > max γα . α∈S1 α∈S1 α∈S0 (11) (12) In SLCM, q is uniquely determined by the structure vector δ. Mathematically, q = arg minα:aα δ |α|, where |·| is the cardinality. By such definition, γq = maxα∈S1 γα = minα∈S1 γα always holds, therefore, to verify (12), we only need to check, γq > max γα . (13) α∈S0 In the following two remarks, we list some observations that are useful in the proof. 2 p2 p Remark 8. 1. L0 L1 = S0 S1 = {α|α α 0 } p p 2. a q, p = 1 ⇒ S1 ⊆ L1 , S0 ⊇ L0 . Remark 9. (t) 1. ∀α, (t+1) γα,− p = γα,− p := γα,− p γα(t+1) = γα(t) (t+1) (t) (t+1) γα = γα − β (t) p + βp p 2. ∀α ∈ L0 , p 3. ∀α ∈ L1 , p 4. ∀α 1 , α 2 ∈ L1 , p 5. ∀α 1 , α 2 ∈ L0 , (t) (t) (t+1) (t+1) γ α1 > γ α2 ⇒ γ α1 > γ α2 (t) (t) (t+1) (t+1) γ α1 > γ α2 ⇒ γ α1 > γ α2 In the following lemma, we give the sufficient and necessary condition for (11). Lemma 6. (Lower bound 1) min γ (t+1) αα 0 α ≥ γα(t+1) =0 0 PSYCHOMETRIKA holds if and only if β (t+1) ≥ maxp (−γα,− p ). p (14) α∈L1 Proof. Since B (t) ∈ B(B), we have minα∈L p γα(t+1) = minα∈L p γα(t) ≥ 0. So we only need to 0 p consider α ∈ L1 , such that 0 (t) minp γα(t+1) = minp (γα,− p + β (t+1) ) ≥ 0, p α∈L1 α∈L1 which holds if and only if (14) holds. (t) We show the relationship between γq(t+1) (t+1) and γq (t) in the following lemma. Lemma 7. (t+1) γq (t+1) = γq (t) ,− p + β (t+1) . p Proof. (t+1) = γq (t) ,− p + β (t+1) . • If q (t+1) = q (t) , γq(t+1) p (t+1) = γq (t) (t) (t+1) • If q (t+1) q (t) , it implies δ p = 0 and δ p (t) (t) = 1. Therefore, q (t) , q (t+1) ∈ S1 , so that, (t) γq (t+1) = γq (t) = γq (t) ,− p . Then, (t+1) γq (t+1) = γq (t+1) ,− p + β (t+1) = γq (t) ,− p + β (t+1) . p p (t+1) = δ (t+1) = 0, and a q (t+1) + e p = a q (t) . Then, • If q (t+1) ≺ q (t) , it means δ (t) p = 1, β p p (t+1) γq(t+1) . (t+1) = γq (t+1) ,− p = γq (t) ,− p = γq (t) ,− p + β p Next, we give the sufficient and necessary condition for (13) in the following lemma. = 1, Lemma 8. (Lower bound 2) Suppose δ (t+1) p (t+1) γq (t+1) > max γα(t+1) , (t+1) α∈S0 if and only if, > maxp γα,− p − γq (t) ,− p . β (t+1) p α∈L0 (15) YINYIN CHEN ET AL. (t+1) (t+1) p Proof. Since δ p = 1, by Remark 2, we have S0 ⊇ L0 . It is easy to see that if (12) holds at time t + 1, then (15) holds, because (t+1) > max γα(t+1) ≥ maxp γα(t+1) = maxp γα,− p . γq (t+1) = γq (t) ,− p + β (t+1) p (t+1) α∈L0 α∈S0 α∈L0 Next we show that if (15) holds, then (13) holds at time t + 1. Because (12) holds at time t, we have, (t) γq(t) (t) > max γα ≥ max 3 (t) α∈S0 p α∈L1 (t) S0 γα(t) . (16) Next, we check (13) in two different scenarios. (t) (t+1) • If q (t) = q (t+1) , S0 = S0 , then by (16) and Remark 9.4, we obtain (t+1) (t+1) γq (t+1) = γq (t) > max 3 (t+1) S0 p α∈L1 (t) γα(t+1) . (t) (t) • If q (t) ≺ q (t+1) , then since q (t+1) , ∈ S1 , we have γq (t+1) > maxα∈L p 3 S(t) γα . By 1 0 Remark 9.4, we have (t+1) γq (t+1) > max 3 p α∈L1 (t) S0 γα(t+1) . On the other hand, since δ (t+1) = δ (t) + e p , {α|α ∈ S(t+1) ,α ∈ / S(t) 0 0 } = {α|α δ (t) , α δ (t+1) } ⊆ L0 = (L1 )c , p p leading to, p L1 4 (t+1) S0 p = L1 4 (t) S0 . (t+1) Proof of Proposition 1. Suppose δ p hold at time t + 1, if = 1, by Lemmas 6 and 8, the monotonicity constraints ! β (t+1) p 1 > max maxp (−γα,− p ), maxp γα,− p − γq (t) ,− p α∈L1 α∈L0 := max(L 1 , L 2 ) = L . In the following two lemmas, we discuss the flipping rule of δ p . (t) (t+1) Lemma 9. (Flipping rule 1) If δ p = 0, δ p t + 1 and L = 0. = 0, the monotonicity constraints hold at time PSYCHOMETRIKA Proof. The monotonicity constraints hold at time t + 1, because B (t) = B (t+1) and B (t) satisfy the constraints. (t) • L 1 = − minα∈L p γα ≤ 0 because (11) holds at t. 1 (t) (t) • L 2 = maxα∈L p γα − γq (t) = 0 because 0 (t) (t) γq (t) = min γα(t) ≤ maxp γα(t) ≤ max γα(t) = γq (t) , (t) p since L0 3 α α∈L0 α∈S1 (t) S1 is not empty. Therefore, L = max(L 1 , L 2 ) = 0. (t) (t+1) Lemma 10. (Flipping rule 2) Suppose δ p = 1, δ p at time t + 1 if L ≤ 0. = 0. The monotonicity constraints hold Proof. If q (t) = q (t+1) , the statement can be proved easily by Lemma 6 and Lemma 8. We check (11) and (13) in for the case that q (t) q (t+1) . (t+1) (t+1) ≤ 0 and minα∈L p γα • Since L 1 = − minα∈L p γα 1 0 at t + 1. p 3 (t+1) • By Remark 9.4, for any α ∈ L1 S0 , (t+1) (t+1) γq (t+1) = γq (t) (t+1) (t) = minα∈L p γα ≥ 0, (11) holds > γα(t+1) 0 (17) (t+1) Together with L 2 = maxα∈L p γα − γq (t+1) ≤ 0, (17) holds for any α ∈ 0 p 2 p 3 (t+1) L0 (L1 S0 ). p ,α ∈ / S(t) Further, as shown in the proof of Lemma 8, we have {α|α ∈ S(t+1) 0 0 } ⊆ L0 , such 2 3 p p (t+1) (t+1) that S0 ⊆ L0 (L1 S0 ). References Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88(422), 669–679. Allman, E. S., Matias, C., & Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics, 37, 3099–3132. Carreira-Perpiñán, M., & Renals, S. (2000). Practical identifiability of finite mixtures of multivariate Bernoulli distributions. Neural Computation, 12, 141–152. Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of the DINA Q-matrix. Psychometrika, 83, 89–108. Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866. Chiu, C.-Y., Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74(4), 633–665. Cox, D., Little, J., O’Shea, D., & Sweedler, M. (1994). Ideals, varieties, and algorithms. American Mathematical Monthly, 101(6), 582–586. Culpepper, S. A. (2019). Estimating the cognitive diagnosis Q matrix with expert knowledge: Application to the fractionsubtraction dataset. Psychometrika, 84, 333–357. Dang, N. V. (2015). Complex powers of analytic functions and meromorphic renormalization in QFT. arXiv preprint arXiv:1503.00995. Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report Series, 2005(2) de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179–199. YINYIN CHEN ET AL. de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333–353. DiBello, L. V., Stout, W. F., & Roussos, L. A. (1995). Unified cognitive/psychometric diagnostic assessment likelihoodbased classification techniques. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively Diagnostic Assessment, Chapter 15 (pp. 361–389). New York: Routledge. Fang, G., Liu, J., & Ying, Z. (2019). On the identifiability of diagnostic classification models. Psychometrika, 84(1), 19–40. Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61(2), 215–231. Gyllenberg, M., Koski, T., Reilink, E., & Verlaan, M. (1994a). Non-uniqueness in probabilistic numerical identification of bacteria. Journal of Applied Probability, 31(2), 542–548. Gyllenberg, M., Koski, T., Reilink, E., & Verlaan, M. (1994b). Non-uniqueness in probabilistic numerical identification of bacteria. Journal of Applied Probability, 31(2), 542–548. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26(4), 301–321. Hagenaars, J. A. (1993). Loglinear models with latent variables (Vol. 94). Newbury Park, CA: Sage Publications Inc. Hartz, S.M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Ph.D. thesis, University of Illinois at Urbana-Champaign. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272. Kruskal, J. B. (1976). More factors than subjects, tests and treatments: An indeterminacy theorem for canonical decomposition and individual differences scaling. Psychometrika, 41(3), 281–293. Kruskal, J. B. (1977). Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and Its Applications, 18(2), 95–138. Liu, J., Xu, G., & Ying, Z. (2013). Theory of the self-learning q-matrix. Bernoulli, 19(5A), 1790–1817. Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64(2), 187–212. Mityagin, B. (2015). The zero set of a real analytic function. arXiv preprint arXiv:1512.07276. Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford Press. Tatsuoka, C. (2002). Data analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society, Series C (Applied Statistics), 51(3), 337–350. Tatsuoka, K. K. (1984). Analysis of errors in fraction addition and subtraction problems. Final report, Technical report. Teicher, H., et al. (1961). Identifiability of mixtures. The Annals of Mathematical Statistics, 32(1), 244–248. Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287. Xu, G. (2017). Identifiability of restricted latent class models with binary responses. The Annals of Statistics, 45(2), 675–707. Xu, G., & Shang, Z. (2017). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113, 1284–1295. Yakowitz, S. J., & Spragins, J. D. (1968). On the identifiability of finite mixtures. The Annals of Mathematical Statistics, 39, 209–214. Manuscript Received: 10 AUG 2018 Final Version Received: 13 DEC 2019 J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 Contents lists available at SciVerse ScienceDirect Journal of Behavior Therapy and Experimental Psychiatry journal homepage: www.elsevier.com/locate/jbtep Effects of depression on reward-based decision making and variability of action in probabilistic learning Yoshihiko Kunisato a, Yasumasa Okamoto b, c, Kazutaka Ueda d, Keiichi Onoda e, Go Okada b, f, Shinpei Yoshimura b, g, Shin-ichi Suzuki a, Kazuyuki Samejima h, Shigeto Yamawaki b, c, * a Faculty of Human Sciences, Waseda University, 2-579-15, Mikajima, Tokorozawa, Saitama 359-1192, Japan Department of Psychiatry and Neurosciences, Graduate School of Biomedical & Health Sciences, Hiroshima University, 1-2-3, Kasumi, Minami-ku, Hiroshima-city, Hiroshima 734-8551, Japan c Core Research for Evolutional Science and Technology, 5, Sanbancho, Chiyoda-ku, Tokyo 102-0075, Japan d Graduate School of Engineering, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-8656, Japan e Department of Neurology, Shimane University, 89-1, Enya-cho, Izumo-city, Shimane 693-8501, Japan f Department of Psychiatry, University of Michigan, 4250 Plymouth Road, Ann Arbor, MI 48109, USA g Research Fellow of the Japan Society for the Promotion of Science, 6, Ichibancho, Chiyoda-ku, Tokyo 102-8471, Japan h Tamagawa University Brain Science Institute, Graduate School of Brain Sciences, 6-1-1, Tamagawagakuen, Machida-city, Tokyo 104-8610, Japan b a r t i c l e i n f o a b s t r a c t Article history: Received 28 October 2011 Received in revised form 22 May 2012 Accepted 22 May 2012 Background and objectives: Depression is characterized by low reward sensitivity in behavioral studies applying signal detection theory. We examined deficits in reward-based decision making in depressed participants during a probabilistic learning task, and used a reinforcement learning model to examine learning parameters during the task. Methods: Thirty-six nonclinical undergraduates completed a probabilistic selection task. Participants were divided into depressed and non-depressed groups based on Center for Epidemiologic Studies eDepression (CES-D) cut scores. We then applied a reinforcement learning model to every participant’s behavioral data. Results: Depressed participants showed a reward-based decision making deficit and higher levels of the learning parameter s, which modulates variability of action selection, as compared to non-depressed participants. Highly variable action selection is more random and characterized by difficulties with selecting a specific course of action. Conclusion: These results suggest that depression is characterized by deficits in reward-based decision making as well as high variability in terms of action selection. Ó 2012 Elsevier Ltd. All rights reserved. Keywords: Depression Reward sensitivity Reinforcement learning Decision making Probabilistic learning Variability of action selection 1. Introduction Evidence suggests that depression is associated with hyposensitivity to reward (Eshel & Roiser, 2010). Depression is characterized by low positive affect derived from pleasant events and high negative affect in response to unpleasant events (Brown, Chorpita, & Barlow, 1998; Kasch, Rottenberg, Arnow, & Gotlib, 2002; McFarland, Shankman, Tenke, Bruder, & Klein, 2006; Watson et al., 1995). In laboratory behavioral studies, a signal detection theory approach has been used to examine the decreased reward * Corresponding author. Department of Psychiatry and Neurosciences, Graduate School of Biomedical & Health Sciences, Hiroshima University, 1-2-3, Kasumi, Minami-ku, Hiroshima 734-8551, Japan. Tel.: þ81 82 257 5208; fax: þ81 82 257 5209. E-mail address: [email protected] (S. Yamawaki). 0005-7916/$ e see front matter Ó 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.jbtep.2012.05.007 sensitivity said to be associated with depression. Henriques, Glowacki, and Davidson (1994) carried out a verbal recognition task that featured three conditions (no monetary feedback, monetary reward and monetary punishment), using signal detection theory to examine the effects of depression on recognition sensitivity and response bias. In their task, participant responses were more liberal in the reward and punishment conditions as compared to the neutral condition. Liberal responding in this context means optimizing responding so as to increase earnings rather than to make correct responses per se, such that a liberal response bias leads to increased earnings. Compared to nondepressed participants, both subclinically (Henriques et al., 1994) and clinically depressed participants (Henriques & Davidson, 2000) evidence less liberal response bias during a reward condition but not punishment or neutral conditions. Pizzagalli, Jahn, and O’Shea (2005) developed another signal detection task that was designed Y. Kunisato et al. / J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 to examine responsiveness to rewards. In their probabilistic reward task, participants were asked to identify whether the mouth length of a cartoon face was short or long, with participants receiving asymmetrical rewards depending on whether correct responses were provided. Non-depressed participants learned a response bias in which the highly reinforced choice was preferred, while nonclinical depressed participants and MDD patients showed low levels of response bias (Pizzagalli, Iosifescu, Hallett, Ratner, & Fava, 2008; Pizzagalli et al., 2005). These researchers have postulated that depression may impair the integration of reinforcement history (Pizzagalli et al., 2008). These earlier studies have linked depression with hyposensitivity to reward during recognition and discrimination tasks. However, some questions remain. The first is whether depression affects hyposensitivity to rewards in probabilistic learning that mixes reward and punishment contingencies. Although earlier studies have examined sensitivity to reward and punishment independently, in our daily lives we encounter many situations in which both reward and punishment are potential outcomes. For example, depressed people often avoid social gatherings in order to avoid having negative experiences in these situations (increased avoidance of punishment), at the potential cost of obtaining social rewards (decreased approach of reward). Most behaviors involve possible positive and negative outcomes, and it is not clear whether depressed individuals consistently exhibit hyposensitivity to reward in such learning scenarios; this possibility needs to be examined. The second question concerns whether depression does indeed affect the integration of reinforcement history. It has been postulated (based on signal detection studies) that depression is associated with impairments in the integration of reinforcement history over time (Pizzagalli et al., 2008), but such integration is not directly observable. Integration is not a direct measure of learning per se but is instead a learning parameter that modulates outcomes. Investigating an observable learning parameter that corresponds to the integration of reinforcement history is necessary for a complete understanding of learning impairments in depression. In an attempt to answer these questions, we used a probabilistic selection task (Frank, Seeberger, & O’Reilly, 2004) along with a reinforcement learning model to analyze the results (Behrens, Woolrich, Walton, & Rushworth, 2007; Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006; Frank, Moustafa, Haughey, Curran, & Hutchison, 2007; O’Doherty et al., 2004; Samejima, Ueda, Doya, & Kimura, 2005; Tanaka et al., 2004). The probabilistic selection task is used to determine the degree to which participants’ decisions are based on reward and/or punishment during probabilistic learning. In this task (Frank et al., 2004; Frank, Woroch, & Curran, 2005), three stimulus pairs (AB pair, CD pair, EF pair) are presented in a random order. Participants have to choose one of two stimuli and are probabilistically reinforced via positive and negative feedback (correct or incorrect). A choice of stimulus A for the AB pair is associated with an 80% likelihood of positive feedback (and a corresponding 20% likelihood of negative feedback), whereas a choice of stimulus B leads to a 20% likelihood of positive feedback (and an 80% likelihood of negative feedback). CD and EF pair are less reliable than are AB pair. A choice of stimulus C for the CD pair leads to 70% positive feedback (with stimulus D leading to 30% positive feedback), while a choice of stimulus E for the EF pair leads to 60% positive feedback (stimulus F leads to 40% positive feedback). Participants are trained to choose stimuli A, C, and E more often than B, D, and F. Stimulus A leads to the highest probability of a positive outcome, whereas stimulus B leads to the highest probability of a negative outcome. After the training session, participants are evaluated the extent to which participants learn from positive and negative feedback of their decisions. 1089 Participants are tested with the same training pairs (i.e., AB, CD, EF) and all novel combinations of stimuli (e.g. AC, BE, etc.) in a random order. If participants learned from positive feedback, they should reliably choose stimulus A when presented with novel test pairs. In contrast, if participants learned from negative feedback, they should reliably avoid stimulus B. Modeling participants’ sequential behavioral data from a reinforcement learning standpoint provides a useful approach to examining an observable learning parameter that corresponds to the integration of reinforcement history (Sutton & Barto, 1998). Computational reinforcement learning models can be used to simulate the learning process and have been developed in a machine learning context. Such models allow us to examine internal learning parameters which are not observable via simple descriptions of behavioral data. We adopted the modified Q learning model (Frank et al., 2007) which contains three learning parameters: aG, controlling the updating speed of selected choice value following positive feedback, aL, controlling the updating speed of selected choice value following negative feedback, and s, controlling the degree of exploiting learned choice value. These learning parameters are associated with the integration of reinforcement history over time, in particular s, which represents direct regulation of this process. Chase et al. (2010) used a probabilistic selection task to examine hypersensitivity to punishment in MDD patients, finding no significant differences between MDD patients and healthy controls on task performance. We speculate that the reasons for this null finding include long decision duration (a maximum of 4 s), type of feedback provided, and antidepressant medication effects. MDD patients responded more slowly than did healthy controls (Chase et al., 2010) and patients also show a blunted response to feedback in terms of reaction times during reinforcement learning (Steele, Kumar, & Ebmeier, 2007). These results suggest that long decision durations will be unlikely to permit detection of group differences in probabilistic learning. Additionally, Chase et al. (2010) used verbal feedback (e.g. “correct”, “incorrect”), but it has been reported that monetary feedback enhances behavioral performance relative to non-monetary feedback (Small et al., 2005). A probabilistic selection task study of depression should feature short decision-making durations and the provision of monetary feedback. The objective of the present study was to use a probabilistic selection task to examine depression-related bias on reward or punishment based decision-making, as well as the effects of depression on the learning parameters derived from a computational reinforcement learning model. We hypothesized that depressed participants would show lower sensitivity to reward during probabilistic learning, as compared to non-depressed participants. We also explored possible depression-related impairments in learning parameters related to the integration of reinforcement history. 2. Materials and methods 2.1. Participants Sixty undergraduate student participants (17 men, 43 women; mean age ¼ 19.45 1.43 years) were recruited from Hiroshima University in Japan. Informed written consent was obtained from all participants. All participants completed the Center for Epidemiologic Studies - Depression scale (CES-D) (Radloff, 1977) when they were first recruited to participate in the experiment. Potential participants were ultimately excluded from further data analysis if they did not reach the attainment criterion during the probabilistic selection task, if they reported neurological illness, or if their CES-D score changed to a value over the cut-off point (CES-D score of 16) 1090 Y. Kunisato et al. / J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 from time of recruitment to time of the experiment. One excluded participant was being treated with antidepressant medication. Twelve participants were excluded due to changes in CES-D scores (6 potentially depressed participants and 6 potentially nondepressed participants). Eleven participants were excluded for not meeting the attainment criterion (8 depressed and 3 nondepressed). The final sample consisted of 18 depressed participants (CES-D score >¼ 16, mean age ¼ 19.44 0.98 years) and 18 non-depressed participants (CES-D score < 16, mean age 19.33 1.14 years). The study protocol was approved by the Hiroshima University ethics committee. 2.2. Procedure Before the behavioral task was conducted, participants completed several self-report measures of mood and depressive symptoms. The CES-D and Beck Depression Inventory-II (BDI-II) (Beck, Steer, & Brown, 1996) were used to assess levels of depressive symptoms. The trait form of the Positive and Negative Affect Scales (PANAS) (Watson, Clark, & Tellegen, 1988) was used to assess fundamental affect. We conducted the probabilistic selection task developed by Frank et al. (2004). Participants were verbally instructed in how to perform the task, and were told that the aim of the task was to maximize total gains. The task was presented using Presentation version 7.3 (Neurobehavioral Systems, California, USA) and lasted approximately 45 min. Participants sat in front of a computer screen and chose one of two paired visual stimuli by pressing either the left or right button on a mouse. We used meaningless pictures (Endo, Saiki, Nakao, & Saito, 2003) as visual stimuli. These pictures have been used in previous probabilistic learning studies (Ohira et al., 2009; Ohira et al., 2010). The task consisted of separate training and test phases. Fig. 1 shows the timing of visual stimulus presentation and participant responses. During the training phase, the stimulus events consisted of a fixation period of 500 ms, followed by a stimulus pair presentation for 750 ms, followed by a blank screen for 350 ms, and finally visual feedback for 600 ms (Fig. 1). Three different stimulus pairs (AB, CD, EF) were presented in a random order, such that each stimulus was randomly presented on the right or left side. Feedback was provided after each choice, in the form of a reward (þ10 yen) or punishment (10 yen). This feedback was probabilistic. Choosing A led to a reward on 80% of the AB pairs, and led to punishments (10 yen) on 20% of the AB pairs. Choosing B led to a reward on 20% of the AB pairs, and led to punishment on 80% of the AB pairs. The CD and EF pairs were less predictable. Choosing C led to reward on 70% of the CD pairs, while choosing E led to reward on 60% of the EF pairs. Participants continued the learning phase until they had met the criterion (choosing A over B on 65% of the trials, choosing C over D on 60% of the trials, and choosing E over F on 50% of the trials). Learning achievements were evaluated every block. A block consisted of 60 trials, and participants performed up to a maximum of 10 blocks (600 trials). Reward 600ms feedback Punishment or -10 yen +10 yen 350ms blank 750ms choice 500ms blank 750ms choice 500ms blank trial onset trial onset Learning session Combinations: AB,CD,EF Probability of reward: A(80%) B(20%) C(70%) D(30%) E(60%) F(40%) Test session Combinations: AB, CD, EF (original pair) AC, AD, AE, AF (Choose A) BC, BD, BE, BF (Avoid B) CE, CF, DE, DF (others) Fig. 1. Schematic outlining the probabilistic selection task. During the training phase, participants were presented with the three pair stimulus and learned to make selections such that rewards could be maximized. During the test phase, participants selected the stimulus from the training pair and all novel combinations of stimulus in a random order. Y. Kunisato et al. / J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 Once participants reached the criterion, they were tested with the same training pairs (i.e., AB, CD, EF) and all novel combinations of stimuli (e.g. AC, BE, etc.) in a random order. Each test pair was presented six times, for a total 90 trials. Participants were again instructed to choose stimuli in order to maximize total gains. If they were unsure of which stimulus to choose, they were instructed to use ‘gut instinct’. No feedback was provided during the test phase. The trials in which A (i.e., AC, AD, AE, AF) or B (i.e. BC, BD, BE, BF) was presented in novel pairings were of primary interest in the present task. Because stimulus A was most often associated with positive feedback during training, the more the participant chooses stimulus A, the more the participant learned or benefited from the positive feedback. Similarly, stimulus B was most often associated with negative feedback during training, such that the more the participant avoids stimulus B, the more the participant learned from the negative feedback. In addition, the correct response rate for the original stimulus pairs (i.e. AB, CD, EF) serves as an index of learning efficacy. After the behavioral task, participants completed the post experiment questionnaire (Frank et al., 2004), which assessed awareness judgments. Finally, participants were paid the money they acquired during the task (500yen w 1380yen). 1091 probability using the temperature parameter s, which is defined as randomness of action selection. The temperature parameter s is related to the degree to which learned values are exploited during task performance (i.e., when temperature parameter s becomes large, the selections become more random). For the probability of choosing stimulus A over stimulus B, we applied the following equation: Q A ðtÞ PA ðtÞ ¼ e Q A ðtÞ e s s (2) Q B ðtÞ þe s The same procedure applies for other trial types, replacing A and B with C, D, E, or F. The s parameter regulates the explorationeexploitation parameter of the reinforcement history. The Q-learning model was then fitted to each participant’s behavioral data by searching for the parameter values (aG, aL, s) that resulted in the best fit. Fitness of data was measured using the log likelihood estimation (LLE). The parameters were searched using the MATLAB Optimization Toolbox. LLE is computed as follows: LLE ¼ log Y Pi* ;t (3) t 2.3. Data analysis Between-groups t-tests were conducted on psychological assessment scores (CES-D, BDI-II, PANAS), the number of trials that participants need to reach the criterion, and the expected value of money gained during the test session. The expected value was computed for every selection during the test session via the following equation: (value of positive feedback (10 yen) * probability of positive feedback for selected choice) þ (value of negative feedback (10 yen) * probability of negative feedback for selected choice). Total expected value was then computed by summing all expected values for each selection. Test session measurements consisted of three scores (number of stimulus A choices, number of stimulus B avoidances, and number of correct responses in original training pair during test session). To examine whether the memory maintenance capability between training and test sessions did not differ across the groups, the number of correct responses for the original training pairs during the test session was included in test session measurements. Multivariate ANOVA was used to examine the group effects on these test session measurements. We used a reinforcement learning framework called the Qlearning model to clarify the parameters related to the learning process, including integration of reinforcement history, which is not directly observable on the basis of behavioral data (Sutton & Barto, 1998). In the Q learning models (Sutton & Barto, 1998), the Q value (action value) is computed using the learning rate (a), which controls speed of selected choice value updating. We applied the following modified Q learning model (Frank et al., 2007) to our behavioral data, computing the Q value for selecting each stimulus i during trial t. The updating rule for the Qi value involves Q i ðt þ 1Þ ¼ Q i ðtÞ þ aG ½rðtÞ Q i ðtÞþ þ aL ½rðtÞ Q i ðtÞ 3. Results Participants’ demographic and self-report data are shown in Table 1. Depressed participants reported significantly higher levels of depressive symptoms on the CES-D and the BDI-II than nondepressed participants, as well as higher levels of negative affect on the PANAS-NA. In addition, depressed participants reported significantly lower positive affect on the PANAS-PA than did nondepressed participants. Based on the post experiment questionnaire results (Frank et al., 2004), all participants were judged to be unfamiliar with the experimental stimuli and to have no irrational belief about feedback that would affect the results during the learning or test sessions. There were no significant group differences in the number of trials needed to reach the criterion (depressed: 360.00 146.97, non-depressed: 320 120.00; t(34) ¼ 0.894, p ¼ 0.38, Hedge’s g ¼ 0.29, 95%CI [0.37 0.95]) and the expected monetary value Table 1 Demographics and self-report data for depressed and non depressed participants (1) A for A > 0 A for A < 0 and ½A 0 otherwise 0 otherwise In this model, the Q value is updated using either learning rate for Gain: aG or learning rate for Loss: aL, depending on whether the outcome is better or worse than expected. aG controls the updating speed of selected choice value following positive feedback, whereas aL controls the updating speed of selected choice value following negative feedback. In addition, choice behavior was modeled using the softmax function (Sutton & Barto, 1998). We computed behavior selection where ½Aþ Pi*,t denotes the estimated probability of participant choice on trial t, which is computed via the softmax function (2). The maximum LLE value is the most predictive of participants’ behavioral data. We examined fit of the Q-learning model using the G2 statistic (Busemeyer & Stout, 2002). The G2 statistic is used to compare the log likelihood for Q-learning and baseline models. The baseline model in this case refers to one in which stimuli are chosen randomly (p ¼ 0.5 for each trial). Multivariate ANOVA was conducted to examine group effects on learning parameters. n Female/male Age CES-D (screening) CES-D (experiment) BDI-II PANAS PA PANAS NA Depressed Non depressed t p 18 13/5 19.44 32.11 28.06 21.94 20.44 23.56 18 13/5 19.33 1.14 9.61 3.40 8.72 3.51 4.83 2.92 28.39 6.46 13.11 3.82 0.756 0.10 <.001 3.99 <.001 3.22 <.001 3.55 <.001 1.26 <.001 1.75 0.98 7.21 7.73 6.17 6.12 7.52 0.31 11.98 9.66 10.64 3.79 5.25 Hedge’s g Note. n ¼ number of subject, CES-D ¼ the Center for Epidemiologic Studies Depression scale, BDI-II ¼ Beck Depression Inventory-II, PANAS PA ¼ Positive affect of Positive and Negative Affect Scales, PANAS NA ¼ Negative affect of Positive and Negative Affect Scales. Y. Kunisato et al. / J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 gained during the test session (depressed: 84.67 46.76, nondepressed:109.33 32.07; t(34) ¼ 1.846, p ¼ 0.07, Hedge’s g ¼ 0.62, 95%CI [1.29 0.05]). During the test session, participants correctly choose all relevant stimuli at an above chance level (greater than 50%). For all three test measurements, participants’ performance significantly departed from chance level (Choose A: mean ¼ 63%, 95%CI [57 69], t(35) ¼ 4.65, p < 0.001; Avoid B: mean ¼ 62%, 95%CI [56 69], t(35) ¼ 3.89, p < 0.001; Original pairs: mean ¼ 75%, 95%CI [71 80], t(35) ¼ 12.23, p < 0.001). A multivariate ANOVA (one factor: depression) with the three test measurements as dependent variables (scores of Choose A, Avoid B and original pairs) revealed a significant group difference for choosing A, F(1, 34) ¼ 10.45, p ¼ 0.003, Hedge’s g ¼ 1.08, 95%CI [1.78 0.38]. There were no significant group differences in terms of avoiding B, F(1,34) ¼ 0.10, p ¼ 0.745, Hedge’s g ¼ 0.11, 95%CI [0.76 0.55], and performance on the original pairs, F(1,34) ¼ 0.13, p ¼ 0.717, Hedge’s g ¼ - 0.12, 95% CI [0.78 0.53]. Depressed participants showed a lower propensity to choose A as compared to non-depressed participants (Fig. 2). G2 statistics for both groups were positive (12.59 7.04) and significantly greater than 0 (t (35) ¼ 10.73, p < 0.01). There were no significant log likelihood differences between the groups (depressed: 101.31 47.07, non-depressed: 86.65 36.67; t(34) ¼ 1.04, p ¼ 0.31, Hedge’s g ¼ 0.34, 95%CI [0.32 1.00]). A multivariate ANOVA (one factor: depression) with the three parameters entered as dependent variables showed a main effect of depression on the s (F(1,34) ¼ 4.83, p ¼ 0.035, Hedge’s g ¼ 0.73, 95% CI [0.06 1.41]) but not the aG (F(1,34) ¼ 3.12, p ¼ 0.09, Hedge’s g ¼ 0.59, 95%CI [0.08 1.26]) or aL (F(1,34) ¼ 1.25, p ¼ 0.271, Hedge’s g ¼ 0.37, 95%CI [0.29 1.03]). Depressed participants showed higher s values than non-depressed participants (Fig. 3). Correlational analyses were conducted to examine the relationship between learning parameters and learning and test phase performance (number of trials to reach criterion during the learning session, and performance on the test session as based on choosing stimuli to maximize gain). The learning parameter s correlated positively with the number of trials required to reach criterion (r ¼ 0.40, p < 0.05, 95%CI [0.08 0.64]). There were no significant correlations between learning parameters and test performance. 4. Discussion This was the first depression study to clarify the nature of reward-based decision making deficits during probabilistic Fig. 2. Proportion of choices made during the test session. The proportion of choose A stimulus (a), the proportion of avoid B stimulus (b), the proportion of correct selection in the original pairs (c). Error bars represent 95% confidence intervals. a Coefficient of metaparameters 1092 b 1.4 1.4 depressed non depressed c 1.4 1.2 1.2 1 1 1 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0 0 Gain * 1.2 0 Loss Fig. 3. Effect of depressed mood on the learning rate for gain (a), learning rate for loss (b), and the variability of action choice (c) estimated from training session behavioral data. Error bars represent 95% confidence intervals. learning, as indexed by the learning parameter s. Our results demonstrate that depressed participants benefit less from rewardbased probabilistic learning and show greater variability in action selection than do non-depressed participants. Notably, our depressed and non-depressed groups showed no differences in punishment-based decision making, ability to memorize the original training pairs, and time needed to reach the criterion: Depression selectively affects reward-based decision making. This deficit is likely associated with hyposensitivity to reward, as demonstrated in previous signal detection studies (Henriques & Davidson, 2000; Henriques et al., 1994; Pizzagalli et al., 2008; Pizzagalli et al., 2005). Depression is characterized by anhedonia: An inability to experience pleasant affect (American Psychiatric Association, 2000). We examined both learning and decision-making processes, with our results suggesting that depressed participants appear to have no learning deficits per se, but that they do have a diminished tendency to base their decisions on likelihood of obtaining rewards. Our results support the hypothesis that diminished functioning of an approach-related system plays a key role in depression (Costello, 1983; Depue & Iacono, 1989; Eshel & Roiser, 2010; Loas, 1996). We also found that depressed participants showed higher learning parameter s values than non-depressed participants. This is the first study that has used an observable learning parameter to provide evidence of impaired integration of reinforcement history in depressed participants. The learning parameter s is defined as randomness of action selection, which is related to the degree to which learned values are exploited during task performance. Participants with high s values have trouble deciding upon and selecting a specific action as compared to participants who have low s values, such that action selection is more variable in such cases. Impairments in the integration of reinforcement history represent deficits in the exploitation of learned choice values, such that s is positively correlated with the length of time to achievement of the learning criterion. This suggests that participants with high s tended to spend more time completing the learning, incurring greater costs during this process than did those with low s values. High s values seem to index decision-making deficits that occur after learning has taken place, but such values were not related to reward or punishment based decision-making during the test session. Further studies are needed to clarify the relationship between s values and decision- making at test, along with possible mechanisms. Huys and Dayan (2009) formulated a behavioral control model of depression, based on a Bayesian reinforcement learning model. They argued that depression results from a mismatch between the Y. Kunisato et al. / J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 environment’s characteristics and a participants’ assumptions about them (Huys & Dayan, 2009). The high s values that we observed might reflect a depression-related tendency to assume that one’s environment is more uncertain than is actually the case, such that depressed participants have greater difficulty making choices that should lead to positive outcomes. However, our study does not permit us to make firm statements about the relationship between learning parameter s and specific assumptions about environmental characteristics. Future studies should examine the relationship between learning parameter s and such assumptions using a Bayesian reinforcement learning model and adequate tasks. Our findings are not consistent with those of Chase et al. (2010). In agreement with our prediction, a probabilistic selection task with short durations and monetary feedback is able to detect reward-based decision making deficits in depression. In our study, depressed undergraduates who were not taking antidepressant medication completed a probabilistic selection task. Recent functional neuroimaging studies have shown that antidepressant medication affects areas of the brain reward system such as the striatum (Abler, Gron, Hartmann, Metzger, & Walter, 2012; McCabe & Mishor, 2011; Ossewaarde et al., 2011). In one study, antidepressant medication led to increased activation of the brain reward system of previously unmedicated MDD patients, such that there were no significant differences between medicated MDD patients and healthy controls (Stoy et al., 2012). Antidepressant medication might have influenced the results of the Chase et al. (2010) study, which included individuals taking medication. Further studies of unmedicated MDD patients using a probabilistic selection task may help to resolve the inconsistency in findings. Two limitations of the present study need to be considered. First, our study was conducted on nonclinical undergraduates. This raises the possibility that our results do not directly apply to MDD patients. However, our use of the recommended CES-D cut-off point allowed us to examine reinforcement learning impairments in depressed participants. As described above, our results differ from those of a previous study conducted with MDD patients (Chase et al., 2010). Future studies should examine possible learning parameter impairments in MDD patients, using our or a similar probabilistic selection task. Second, we used short decision-making durations (750 ms). This short duration might have permitted us to identify a significant difference between depressed and non-depressed participants, but psychomotor symptoms may make it difficult for MDD patients to make decisions within a 750 ms timeframe. Further research is needed to elucidate an optimal decision-making timeframe that will permit useful comparisons between MDD patients and healthy controls to be made. 5. Conclusion The present study showed that depressive participants have a deficit in reward-based decision making in probabilistic learning contexts. Learning parameter s values were higher in depressed than in non-depressed participants. These results help to clarify the mechanisms of faulty decision making observed in depression. Understanding learning parameters that characterize depression will shed new light on depression-related impairments in decision making, thereby leading to more effective treatments and contributing to the development of a computational approach to the study of mood disorders. Acknowledgments This work was supported by the Strategic Research Program for Brain Sciences and Grant-in-Aid for Research Activity Start-up 1093 (23830088) from the Ministry of Education, Culture, Sports, Science and Technology of Japan. References Abler, B., Gron, G., Hartmann, A., Metzger, C., & Walter, M. (2012). Modulation of frontostriatal interaction aligns with reduced primary reward processing under serotonergic drugs. The Journal of Neuroscience, 32, 1329e1335. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washington, DC: American Psychiatric Association. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck depression inventory manual (2nd ed.). San Antonio, Texas: The Psychological Corporation. Behrens, T. E., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214e1221. Brown, T. A., Chorpita, B. F., & Barlow, D. H. (1998). Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology, 107, 179e192. Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: decomposing performance on the Bechara gambling task. Psychological Assessment, 14, 253e262. Chase, H. W., Frank, M. J., Michael, A., Bullmore, E. T., Sahakian, B. J., & Robbins, T. W. (2010). Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia. Psychological Medicine, 40, 433e440. Costello, E. J. (1983). Information processing for decision making in depressed women. A study of subjective expected utilities. Journal of Affective Disorders, 5, 239e251. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876e879. Depue, R. A., & Iacono, W. G. (1989). Neurobehavioral aspects of affective disorders. Annual Review of Psychology, 40, 457e492. Endo, N., Saiki, J., Nakao, Y., & Saito, H. (2003). Perceptual judgments of novel contour shapes based on their hierarchical descriptions of geometrical properties. Japanese Journal of Psychology, 74, 346e353. Eshel, N., & Roiser, J. P. (2010). Reward and punishment processing in depression. Biological Psychiatry, 68, 118e124. Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences of the United States of America, 104, 16311e16316. Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306, 1940e1943. Frank, M. J., Woroch, B. S., & Curran, T. (2005). Error-related negativity predicts reinforcement learning and conflict biases. Neuron, 47, 495e501. Henriques, J. B., & Davidson, R. J. (2000). Decreased responsiveness to reward in depression. Cognition & Emotion, 14, 711e724. Henriques, J. B., Glowacki, J. M., & Davidson, R. J. (1994). Reward fails to alter response bias in depression. Journal of Abnormal Psychology, 103, 460e466. Huys, Q. J., & Dayan, P. (2009). A Bayesian formulation of behavioral control. Cognition, 113, 314e328. Kasch, K. L., Rottenberg, J., Arnow, B. A., & Gotlib, I. H. (2002). Behavioral activation and inhibition systems and the severity and course of depression. Journal of Abnormal Psychology, 111, 589e597. Loas, G. (1996). Vulnerability to depression: a model centered on anhedonia. Journal of Affective Disorders, 41, 39e53. McCabe, C., & Mishor, Z. (2011). Antidepressant medications reduce subcorticalcortical resting-state functional connectivity in healthy volunteers. NeuroImage, 57, 1317e1323. McFarland, B. R., Shankman, S. A., Tenke, C. E., Bruder, G. E., & Klein, D. N. (2006). Behavioral activation system deficits predict the six-month course of depression. Journal of Affective Disorders, 91, 229e234. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452e454. Ohira, H., Fukuyama, S., Kimura, K., Nomura, M., Isowa, T., Ichikawa, N., et al. (2009). Regulation of natural killer cell redistribution by prefrontal cortex during stochastic learning. NeuroImage, 47, 897e907. Ohira, H., Ichikawa, N., Nomura, M., Isowa, T., Kimura, K., Kanayama, N., et al. (2010). Brain and autonomic association accompanying stochastic decision-making. NeuroImage, 49, 1024e1037. Ossewaarde, L., Verkes, R. J., Hermans, E. J., Kooijman, S. C., Urner, M., Tendolkar, I., et al. (2011). Two-week administration of the combined serotoninnoradrenaline reuptake inhibitor duloxetine augments functioning of mesolimbic incentive processing circuits. Biological Psychiatry, 70, 568e574. Pizzagalli, D. A., Iosifescu, D., Hallett, L. A., Ratner, K. G., & Fava, M. (2008). Reduced hedonic capacity in major depressive disorder: evidence from a probabilistic reward task. Journal of Psychiatric Research, 43, 76e87. Pizzagalli, D. A., Jahn, A. L., & O’Shea, J. P. (2005). Toward an objective characterization of an anhedonic phenotype: a signal-detection approach. Biological Psychiatry, 57, 319e327. Radloff, L. S. (1977). The CES-D scale: a self-report depression scale for research in the General Population. Applied Psychological Measurement, 1, 385e401. 1094 Y. Kunisato et al. / J. Behav. Ther. & Exp. Psychiat. 43 (2012) 1088e1094 Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005). Representation of actionspecific reward values in the striatum. Science, 310, 1337e1340. Small, D. M., Gitelman, D., Simmons, K., Bloise, S. M., Parrish, T., & Mesulam, M. M. (2005). Monetary incentives enhance processing in brain regions mediating top-down control of attention. Cerebral Cortex, 15, 1855e1865. Steele, J. D., Kumar, P., & Ebmeier, K. P. (2007). Blunted response to feedback information in depressive illness. Brain, 130, 2367e2374. Stoy, M., Schlagenhauf, F., Sterzer, P., Bermpohl, F., Hagele, C., Suchotzki, K., et al. (2012). Hyporeactivity of ventral striatum towards incentive stimuli in unmedicated depressed patients normalizes after treatment with escitalopram. Journal of Psychopharmacology, 26, 677e688. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press. Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neuroscience, 7, 887e893. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of Personality and Social Psychology, 54, 1063e1070. Watson, D., Weber, K., Assenheimer, J. S., Clark, L. A., Strauss, M. E., & McCormick, R. A. (1995). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales. Journal of Abnormal Psychology, 104, 3e14. Journal of Mathematical Psychology 66 (2015) 59–69 Contents lists available at ScienceDirect Journal of Mathematical Psychology journal homepage: www.elsevier.com/locate/jmp The relation between reinforcement learning parameters and the influence of reinforcement history on choice behavior Kentaro Katahira Department of Psychology, Graduate School of Environmental Studies, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Aichi, 464-8601, Japan highlights • • • • Reinforcement learning (RL) models and regression models have been used for choice data analysis. We investigated the relation between these two approaches. We found a special case in which an RL model is equivalent to a regression model. Based on the relation, we discuss how the RL parameters are related to history dependence. article info Article history: Received 7 November 2014 Received in revised form 19 February 2015 Available online 15 May 2015 Keywords: Reinforcement learning History dependence Regression model Model-based analysis abstract Reinforcement learning (RL) models have been widely used to analyze the choice behavior of humans and other animals in a broad range of fields, including psychology and neuroscience. Linear regressionbased models that explicitly represent how reward and choice history influences future choices have also been used to model choice behavior. While both approaches have been used independently, the relation between the two models has not been explicitly described. The aim of the present study is to describe this relation and investigate how the parameters in the RL model mediate the effects of reward and choice history on future choices. To achieve these aims, we performed analytical calculations and numerical simulations. First, we describe a special case in which the RL and regression models can provide equivalent predictions of future choices. The general properties of the RL model are discussed as a departure from this special case. We clarify the role of the RL-model parameters, specifically, the learning rate, inverse temperature, and outcome value (also referred to as the reward value, reward sensitivity, or motivational value), in the formation of history dependence. © 2015 The Author. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). 1. Introduction Reinforcement learning (RL) models have been widely used to analyze the choice behavior of living systems in a wide range of behavioral studies, including psychology and neuroscience (Corrado & Doya, 2007; Daw, 2011; O’Doherty et al., 2004; O’Doherty, Hampton, & Kim, 2007; Yechiam, Busemeyer, Stout, & Bechara, 2005). Evidence of the neural correlates for the subcomponents assumed in RL theory (e.g., reward prediction error, action value) provides the validity to perform an RL model-based analysis for choice behavior (Niv, 2009; Samejima, Ueda, Doya, & Kimura, 2005; Schultz, 1997). An essential feature of the RL model is the formulation of what action to take based on previous experiences of reward or punishment regarding the action. In addition, the linear regression-based E-mail address: [email protected]. approach, using reward history and choice history as explanatory variables and future choice as an objective variable, has also been used to model choice behavior (Corrado, Sugrue, Seung, & Newsome, 2005; Katahira, Fujimura, Okanoya, & Okada, 2011; Kovach et al., 2012; Lau & Glimcher, 2005; Seo, Barraclough, & Lee, 2009; Seo & Lee, 2009; Seymour, Daw, Roiser, Dayan, & Dolan, 2012; Sugrue, Corrado, & Newsome, 2004). The linear regression approach is useful for estimating how reward and choice histories influence future action (e.g., how much influence the reward from n trials ago has on future actions). However, the relation between the RL model and regression models has not been explicitly addressed. Specifically, to what extent and how the predictions differ between the two models has not been explored. Hence, the dependence on reward history in RL models has not been clearly described. In the present study, we aimed to clarify the relation between the parameters of the RL model and the influence of reward history on future choice (specifically, the regression coefficients of the logistic regression models). Because the regression model can http://dx.doi.org/10.1016/j.jmp.2015.03.006 0022-2496/© 2015 The Author. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4. 0/). 60 K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 directly represent the dependency on the reward and choice histories, investigating the relationship between RL-model parameters and regression models would provide valuable information about which behavioral factors may underlie the differences in the model parameters. Conversely, using the relation, one can predict which types of the behavioral sequences can be expected given a specific set of model parameters. To achieve these aims, we performed analytical calculations and numerical simulations. We focused on fundamental RL-model parameters: the learning rate, the outcome value (also referred to as the reward value, reward sensitivity, or motivational value), and the inverse temperature (also referred to as the exploration parameter). These parameters have been used to characterize how psychological factors or personality traits of individuals affect choice behavior (Katahira, Fujimura, Matsuda, Okanoya, & Okada, 2014; Katahira et al., 2011; Kunisato et al., 2012; Lindström, Selbing, Molapour, & Olsson, 2014). However, how these parameters are related to particular behavioral aspects has not been explored sufficiently. The present study will aid in the interpretation of the different impacts of the RL-model parameters. In the present study, we focus on probabilistic learning tasks (also called bandit problems), in which a decision-maker must choose between a set of options, each with different unknown reward rates, to maximize the total reward. The reward rates can dynamically change during the task, but they do not depend on past choices. Such probabilistic learning tasks have been widely used in psychology and neuroscience research. Simplified Q-learning models have often been used in RL model-based analysis of data obtained using this task. A general Q-learning model computes the action value, which is an expected future reward, for each ‘‘state’’ (Watkins & Dayan, 1992). However, for the probabilistic learning tasks that we consider here, there is only one state, and thus, a state variable is not required. Thus, in this study, we focus on a simplified Q-learning model without a state variable, and we will refer to this model as simply the ‘‘Q-learning model’’. In this paper, we first introduce several variants of Q-learning models for probabilistic learning tasks. Next, we describe a logistic regression model, which is a typical regression model used to analyze choice data. Among the variants of the Q-learning models, we find that the forgetting Q-learning model (F-Q model), in which the value of an unchosen option decays by the same amount as the value of chosen, non-rewarded option, is able to make predictions equivalent to those of the logistic regression model. We can view the general Q-learning model as a model that deviates from this special case. The deviation clarifies the special properties of standard RL models. We then present numerical simulation results that demonstrate the relation between the parameters in Q-learning models and the history dependence of choice. Finally, we discuss several implications of our results. Based on the set of action values, the model computes the probability of choosing option 1 using the soft max function: P (a(t ) = 1) = = exp (β Q1 (t )) (1) exp (β Q1 (t )) + exp (β Q2 (t )) 1 1 + exp (−β [Q1 (t ) − Q2 (t )]) , (2) where β is the inverse temperature parameter that determines the sensitivity of the choice probabilities to difference in values. The model subsequently evaluates the outcome of the action. The outcome value in trial t is denoted by R(t ). We typically simply set the binary value for R(t ) such that R(t ) = 1 if a reward is given and R(t ) = 0 if no reward is given. The impact of different outcomes may be quantified by choosing parameters R(t ) = κ1 if outcome 1 is given, R(t ) = κ2 if outcome 2 is given, and R(t ) = 0 if a control outcome is given (Katahira et al., 2014, 2011, 2015). Based on the outcome, the action values for the chosen option i are updated as follows: Qi (t + 1) = Qi (t ) + αL (R(t ) − Qi (t )) , (3) where αL is the learning rate that determines how much the model updates the action value depending on the reward prediction error, R(t ) − Qi (t ). For the unchosen option j (i ̸= j), the action value is updated as follows: Qj (t + 1) = Qj (t ) − αF Qj (t ) (4) = (1 − αF )Qj (t ), (5) where αF is the forgetting rate (Ito & Doya, 2009). In a common RL model-based analysis, the action value of the unchosen option is not typically updated. This convention can be represented by setting αF = 0. We call this the standard Q-learning model. In this study, the forgetting rate parameter plays an important role in the identification of the connection between the regression and RL models, as discussed later. 2.2. Linear regression models Next, we will introduce a regression model that predicts a choice from the reward and choice history of previous trials (Corrado et al., 2005; Lau & Glimcher, 2005; Sugrue et al., 2004). Here, we consider a binary outcome case such that R(t ) = 1 when the reward is given and R(t ) = 0 when no reward is given. Following the convention of Corrado et al. (2005) and Lau and Glimcher (2005), we represent the reward history r (t ) as follows: 1 r (t ) = −1 0 if option 1 is chosen and a reward is given at trial t , if option 2 is chosen and a reward is given at trial t , if no reward is given at trial t . We represent the choice history c (t ) as follows: 1 if option 1 is chosen at trial t , if option 2 is chosen at trial t . 2. Models c (t ) = 2.1. Reinforcement learning models With these history variables, the regression model is defined with a predictor: Here, we introduce an RL model (Sutton & Barto, 1998). Specifically, we consider the Q-learning model (Watkins & Dayan, 1992), which is the most commonly used model for model-based analysis of choice behavior. Throughout the paper, we consider a case with only two options; however, our results can be generalized to multiple-option cases. The model assigns each action, i, an action value, Qi (t ), where t is the index of the trial. In the default setting, the initial action values, Qi (1), are set to zero, i.e., Q1 (1) = Q2 (1) = 0. Let a(t ) ∈ {1, 2} denote the option that was chosen at trial t. h(t ) = −1 Mr m=1 br (m)r (t − m) + Mc bc (m)c (t − m), (6) m=1 where br (m) and bc (m) are the regression coefficients for the trial m trials ago. The constants Mr and Mc are the history length for the reward history and the choice history (from the past trials to the current trial), respectively. Sugrue et al. (2004) and Corrado et al. (2005) used a linear regression approach with an identitylink function and optimized the regression coefficients so that they K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 minimized the sum of squared errors between h(t ) and c (t ). Here, we consider logistic regression with the logit link function (Kovach et al., 2012; Lau & Glimcher, 2005; Seymour et al., 2012)1 : 1 P (a(t ) = 1) = 1 + exp(−h(t )) . (7) We estimate the regression coefficients using the maximum likelihood method. The linear and logistic regressions do not yield qualitatively different results. 3. Analytical calculation 3.1. General formulation To analyze the relation between the Q-learning and logistic regression models, we transform the update rules (Eqs. (3) and (5)) that were originally in the form of a recurrence formula. First, we define the following quantity δt ,i = 1 0 if a(t ) = i if a(t ) ̸= i. (8) With this expression, the update rules (Eqs. (3) and (5)) can be summarized as Qi (t + 1) = Qi (t ) + αt∗,i δt ,i R(t ) − Qi (t ) , 61 β as a free parameter. When different values are assigned to different outcomes, such as R(t ) = κ1 if outcome 1 is given, R(t ) = κ2 if outcome 2 is given, and R(t ) = 0 if the control outcome is given, the relative values among the different outcomes are meaningful; however, the absolute value is not meaningful. This is not the case if the initial value is non-zero, i.e., Qi (1) ̸= 0. However, if αL is sufficiently larger than 0 and if there are sufficient trials, the impact of Qi (1) becomes weak compared with the other terms, and it can be neglected. Taken together, as long as we consider the binary reward history (0: absence of reward, 1: existence of reward) and the case in which the initial value is negligible, varying β and the outcome value κ has the same effect on the reward history dependence of choice probability. 3.2. Special case of the Q-learning model (the forgetting Q-learning model) Here, we consider the special case where the forgetting rate is identical to the learning rate, i.e., αL = αF (Barraclough, Conroy, & Lee, 2004; Ito & Doya, 2009). This model is referred to as the forgetting Q-learning model (F-Q model; Ito & Doya, 2009). For this case, αt∗,i = αL , and Eq. (11) becomes Qi (t + 1) = (1 − αL )t Qi (1) (9) + αL where t −1 (1 − αL )k−1 δt −k,i R(t − k). (12) k=0 αt∗,i = αL δt ,i + αF (1 − δt ,i ). (10) Expanding this update rule back into the past, Qi (t + 1) = Qi (t ) + αt∗,i δt ,i R(t ) − Qi (t ) ∆Q (t + 1) = (1 − αL )t [Q1 (1) − Q2 (1)] = (1 − αt∗,i )Qi (t ) + αt∗,i δt ,i R(t ) = (1 − αt∗,i ) (1 − αt∗−1,i )Qi (t − 1) + αt∗−1,i δt −1,i R(t − 1) + αt∗,i δt ,i R(t ) + αL k=0 ∆Q (t + 1) = αL k=0 y(t ) = Mr j =1 t −1 + k=0 αt∗−k,i δt −k,i br (m) δt −m,1 R(t − m) − δt −k,2 R(t − m) m=1 (1 − αj∗,i ) Qi (1) k−1 (15) Furthermore, the logistic regression model (Eq. (6)) can be written as we obtain Qi (t + 1) = t −1 (1 − αL )k−1 × δt −k,1 R(t − k) − δt −k,2 R(t − k) . + (1 − αt∗,i )αt∗−1,i δt −1,i R(t − 1) + αt∗,i δt ,i R(t ) = ···, (1 − αL )k−1 δt −k,1 R(t − k) − δt −k,2 R(t − k) . (14) If the difference in the initial values of the Q s is zero, i.e., Q1 (1) − Q2 (1) = 0, or it is negligible, Eq. (14) is reduced to + (1 − αt∗,i )αt∗−1,i δt −1,i R(t − 1) + αt∗,i δt ,i R(t ) = (1 − αt∗,i )(1 − αt∗−1,i ) (1 − αt∗−2,i )Qi (t − 2) + αt∗−2,i δt −2,i R(t − 2) t (13) t −1 = (1 − αt∗,i )(1 − αt∗−1,i )Qi (t − 1) From the softmax function (Eq. (2)), we notice that the difference in Q-values between the two options (∆Q (t ) ≡ Q1 (t ) − Q2 (t )) affects the choice probability. From Eq. (12), for ∆Q , we obtain + (1 − αt∗−l,i ) R(t − k). (11) l=0 Note that all terms, with the exception of the first term, are of the first degree in R. Thus, if all initial values of Qi are zero, i.e., Qi (1) = 0 for all i, the transformation R → a × R with a constant a has an equivalent effect on choice probability as the transformation Q → a × Q or β → a × β . Thus, scaling the reward magnitude R by a factor a would not affect the choice probability (Eq. (2)) if β is scaled by 1/a at the same time. This suggests that it is redundant to parameterize the outcome values such that R(t ) = 0 when no reward is given and R(t ) = κ when a reward is given while leaving bc (m) δt −m,1 − δt −m,2 . (16) i=1 A comparison of Eqs. (15) and (16) indicates that these equations are equivalent if we neglect the cutoff effect (the effect of neglecting the terms for more than Mr trials ago), with the following relation br (1) = βαL , br (2) = βαL (1 − αL ), br (3) = βαL (1 − αL )2 , . . . , br (Mr ) = βαL (1 − αL )Mr −1 , bc (1) = bc (2) = · · · = bc (Mc ) = 0. (17) The sum of the coefficients for the reward history, m br (m), converges to β regardless of the learning rate αL if the cutoff effect can be neglected because ∞ 1 This model is also called a conditional logit model. Mc m=1 br (m) = β ∞ m=0 αL (1 − αL )m = β. (18) 62 K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 3.3. Standard Q-learning model (with the forgetting rate αF = 0) The standard learning model in which the action value for the unchosen option is not updated is represented by setting αF = 0 in Eq. (9). With this setting, αt∗,i = αL δt ,i Qi (t + 1) = (1 − αL δj,i ) Qi (1) t −1 + αL δt −k,i k=0 t2 (1 − τ )m−1 δt −m,i . (24) m=1 (1 − αL δt −l,i ) R(t − k). br (2) = βαL (1 − αL ), . . . , br (Mr ) = βαL (1 − αL )Mr −1 , (19) l =0 Denoting the number of trials in which option i is chosen in the trials from trial t1 to trial t2 as nt 1 , t 2 , i ≡ t −1 br (1) = βαL , j=1 k−1 Ci (t ) = τ For the F-Q model, i.e., αL = αF , substituting this choice trace into Eq. (22) and comparing the resulting equation to Eq. (16), we find that these equations are equivalent if we neglect the cutoff effect, with the following relations: and Eq. (11) becomes t to the weighted sum of the choice history, with exponentially decaying coefficients: bc (1) = βϕτ , bc (2) = βϕτ (1 − τ ), . . . , bc (Mc ) = βϕτ (1 − τ )Mc −1 . (25) 4. Results δ t ,i , t = t1 Eq. (19) becomes Qi (t + 1) = (1 − αL )n1,t ,i Qi (1) t −1 + αL δt −k,i (1 − αL )nt −k+1,t ,i R(t − k). (20) k=0 Regarding ∆Q , we have ∆Q (t + 1) = (1 − αL )n1,t ,1 Q1 (1) − (1 − αL )n1,t ,2 Q2 (1) t −1 + αL δt −k,1 (1 − αL )nt −k+1,t ,1 R(t − k) k=0 t −1 − δt −k,2 (1 − αL ) nt −k+1,t ,2 R(t − k) . (21) k=0 It should be noted that as the factor (1 − αL )nt −k+1,t ,i indicates, the influence of a reward (R = 1) depends on the frequency that the subject chooses the same option after the choice and is no longer a constant; thus, it cannot be precisely mapped onto the regression model. 3.4. The choice-autocorrelation factor It has been shown that humans tend to repeat the recent choices, a tendency called ‘‘choice perseverance’’, ‘‘choice stickiness’’, or ‘‘decision inertia’’. Such a tendency is often explicitly modeled by a choice-autocorrelation factor that is added to the action value as follows (Akaishi, Umeda, Nagase, & Sakai, 2014; Gershman, Pesaran, & Daw, 2009; Huys et al., 2011): P (a(t ) = i) = exp (β[Qi (t ) + ϕ Ci (t )]) 2 , (22) exp (β[Qk (t ) + ϕ Ck (t )]) k=1 where Ci (t ) is a trace that quantifies how frequently the option i was chosen recently. The choice trace weight ϕ is a parameter that controls the tendency to repeat (when positive) or avoid (when negative) recently chosen options. An example of the choice trace is a trace computed using the following update rule (Akaishi et al., 2014): Ci (t + 1) = (1 − τ )Ci (t ) + τ δt ,i (23) where we have defined δt ,i as in Eq. (8) and the initial values are set to zero, i.e., C1 (1) = C2 (1) = 0. This choice trace is equivalent We performed numerical simulations to confirm the validity of the analytical calculations and to investigate the relation between the RL and regression models. Throughout the simulations, we adopted the following procedures. First, we generated the choice data from the Q-learning models that performed hypothetical decision-making tasks. We subsequently fitted the parameters of the logistic regression model to the simulated data using the maximum likelihood method. We set the history length as Mr = 10 and Mc = 10. For the majority of the simulations, the Q-learning models performed a simple, probabilistic learning task, unless otherwise stated. In the probabilistic learning task, one of the options was associated with a higher reward probability, pr , compared with the other option that had a reward probability of 1 − pr . With the probability for the chosen option, the reward was given (R(t ) = κ); otherwise, no reward was given (R(t ) = 0). We used pr = 0.7 and κ = 1.0, unless otherwise stated. After each 50trial block, the contingencies of the two stimuli were reversed, and the model performed 500 trials in total (thus, there were 9 reversals in one session). We generated data for 5,000 sessions per condition, which resulted in 2,500,000 trials per condition. The use of a large data set reduces the estimation error of the regression coefficient. We confirmed that the confidence intervals of all regression coefficients were less than 0.05. Therefore, the statistical estimation error can be neglected in the interpretation of the results. 4.1. Special case of the Q-learning model (F-Q model) We began the simulations by confirming the analytical relation between the Q-learning model with the special case αF = αL (F-Q model) and the logistic regression model. We set the inverse temperature parameter β = 3.0. Fig. 1(A) shows the regression coefficients of the logistic regression that were analytically predicted from Eq. (17) (squares) and those obtained by statistical model fitting to the simulated data (solid lines). We can confirm that these two results almost perfectly agree, which supports the validity of the analytical calculation. Eq. (18) predicts that the total sum of the regression coefficient for reward history is equal to the inverse temperature (here, β = 3.0) independent of the learning rate if the cut-off effect can be neglected. Fig. 1(B) shows that this is the case with the exception of the small learning/forgetting rate case where a negative deviation from the predicted value (=3.0) was observed, particularly for a short history length (e.g., Mr = 10). This exception is because of the cutoff effect of reward history: for the small learning/forgetting rate case, greater than 10 previous trials have an influence on the current choice (Fig. 1(A)). Because these trials were not included in K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 A 63 C B Fig. 1. Simulation and analytical results for the special case in which the learning rate (αL ) and the forgetting rate (αF ) are identical (F-Q model). These two parameters were varied while fixed to the same value (αL = αF ). (A) The regression coefficients for the reward history (top) and the choice history (bottom). The solid line represents the coefficients estimated for the logistic regression model fitted to simulated data generated by the Q-learning models. The squares represent the analytical predictions obtained using Eq. (17). (B) The total sum of the regression coefficients for the reward history (top) and the choice history (bottom), while varying the length of the history included in the regression model (Mr = Mc ). (C) The scatter plot of the predictions regarding the current choice (P (a(t ) = 1)) derived from the Q-learning model and the regression model for varying learning rates (with identical forgetting and learning rates). For (A) and (C), the history lengths Mr and Mc are both set to 10. The other parameters were κ = 1 and β = 3.0. the regression model, the total sum of the regression coefficient became smaller than the inverse temperature. Consistent with this interpretation, increasing the history length Mr reduced the discrepancy (i.e., as Mr increased: 20, 30, and 40). We also examined how well the prediction of choice, i.e., P (a(t ) = 1), of the two models matched. Fig. 1(C) compares the predictions of the Q-learning and regression models. Each dot represents the prediction for one trial. The samples from the first 5 sessions are shown for visibility. If the two models’ predictions perfectly agree, the samples should lie on the diagonal line. We see that for the case with a small learning/forgetting rate (αL = αF = 0.2) and a short history length of Mr = 10, there is a slight difference between the two models because of the cut-off effect of reward history in the regression models. With the exception of this case, the two models exhibited almost perfect agreement, as predicted by the analytical calculation. 4.2. The effects of the departure of the forgetting parameter from the learning rate Next, we consider how the deviation of the forgetting rate αF from the learning rate αL affects the behavior of the Q-learning model. Fig. 2 shows the simulation results with a varying αF and a fixed learning rate, αL = 0.4. It is noteworthy that a dependence on choice history arose (bc ̸= 0). When αF was smaller than 64 K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 B A Fig. 2. The effects of the deviation of the forgetting rate αF from the learning rate αL = 0.4. (A) The regression coefficients for reward history (top) and choice history (bottom). The convention is the same as in Fig. 1(A) with the exception that the squares that represent the analytical predictions are only shown for the case with αF = 0.4. (B) The scatter plot of the prediction regarding the current choice (P (a(t ) = 1)) derived from the Q-learning model and the regression model with a varied forgetting rate and a learning rate fixed at 0.4. The other parameters were κ = 1, and β = 3.0. Table 1 Model comparison between the full regression model (Mr = Mc = 10) and the regression model without choice history (Mr = 10, Mc = 0). Data were derived from the Q-learning model using the same sample for which results are shown in Fig. 2. AIC, Akaike Information Criterion (unit is 1,000); Correlation, Pearson product-moment correlation coefficient between the predictions of the Q-learning and regression models. Model Full model No choice history Criteria Forgetting rate AIC (×10 ) Correlation (r ) AIC (×103 ) Correlation (r ) 3 0.0 0.2 0.4 0.6 0.8 1.0 2506.02 0.979 2559.84 0.955 2290.50 0.998 2297.92 0.995 2296.85 1.0 2296.84 1.0 2331.24 0.999 2335.26 0.997 2362.75 0.995 2375.27 0.990 2390.28 0.990 2413.85 0.977 αL , e.g., the standard Q-learning model (αF = 0), a negative dependence on choice history was identified (bc < 0). In contrast, when αF was larger than αL , a positive dependence on choice history was identified (bc > 0). To examine whether the inclusion of choice history in regression modeling improves the prediction, we computed the Akaike Information Criterion (AIC) for the regression models and correlation coefficients of the Q-learning and regression model predictions (Table 1). The AIC is a measure of the predictive ability of a model, and a smaller AIC score indicates a better prediction. With the exception of the case of αL = αF = 0.4 (the F-Q model), the AICs were smaller for the full model with choice history (Mc = 10) compared with the model without choice history (Mc = 0); these findings indicate that the inclusion of choice history provides a better fit. The full regression model exhibited a larger correlation coefficient between the predictions of the Q-learning and regression models, which indicates that the full model’s predictions agreed with the Q-learning model better than the model without choice history. The reason the dependence on choice history arose is explained as follows. Consider an extreme case where no reward was given in the last Mr trials (R(t − 1) = · · · = R(t − Mr ) = 0). In this case, the regression model in which only reward history is included predicts that the subject chose option 1 with a probability of 0.5, i.e., P (a(t ) = 1) = 0.5. However, this consequence differs from the actual behavior of the Q-learning model. In the Q-learning model with αF < αL , the value of the unchosen option remains unchanged (when αF = 0) or decays slowly compared with the chosen option (when αF > 0). In contrast, the value of the chosen option decays, and the tendency for switching the option increases. Thus, the regression coefficients for the choice history, bc , become negative. When the forgetting rate is larger than the learning rate (αF > αL ), the value of the unchosen option decays faster than the chosen option. Thus, the model tends to favor the same choice that was made in the immediate past and accordingly, bc is a positive value. Although these properties of the Q-learning model can be captured, in part, via the incorporation of choice history in the regression model, it does not make the prediction perfect. This point is clarified by the scatter plot of the prediction P (a(t ) = 1) for the two models (Fig. 2(B)). The greater the difference between αL and αF , the larger the deviation of the prediction of the two models (discrepancy from the diagonal lines). This tendency can also be confirmed by the correlation coefficients in Table 1. As the multiplicative factor (1 − αL )nt −k+1,t ,i in Eq. (21) indicates, the influence of an outcome depends on the frequency that the subject chose the same option after the choice. This influence cannot be exactly expressed by the regression model, which does not explicitly include the number of choices after the outcome. This factor represents a property that the general Q-learning model possesses but regression models cannot represent. For a specific trial, the number of times the same option is chosen before the trial is roughly independent of the predictions of the Q-learning model for the trial unless the prediction of the Q-learning model does not take extreme values (i.e., when P (a(t ) = 1) is close to 0 or 1). Thus, the discrepancies are uniformly distributed around the diagonal line, to a certain extent. When the forgetting rate is larger than the learning rate (i.e., αF = 0.8, 1.0), the predictions of the Q-learning model tend to be distributed within a set of discrete values. This occurs because the value of the unchosen option quickly decays to zero; both options have a chance to be the unchosen option, and it is rare that one option continues to be chosen many times so that K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 A 65 B Fig. 3. The effects of parameters in a standard Q-learning model. (A) The effects of the outcome value parameter κ on the dependence of reward history. The other parameters were set to αL = 0.4, αF = 0, and β = 3.0. (B) The effects of the learning rate αL , where αF = 0, β = 3.0, and κ = 1.0. For each panel, the upper two graphs show the regression coefficients for reward and choice history. The lower two graphs show the total sum of the regression coefficients as a function of the varying parameter. the corresponding Q-value continuously change. Thus, P (a(t ) = 1) is distributed discretely. 4.3. The effects of outcome value and learning rate in the standard Q-learning model We examined the influence of the outcome value parameter κ and the learning rate αL for the standard Q-learning model in which the forgetting rate is zero (αF = 0). As previously discussed, if the initial action values are all zero (i.e., Q1 (1) = Q2 (1) = 0), varying the inverse temperature β has the same impact on the choice as scaling R(t ) by the same factor. Thus, varying κ is equivalent to varying β by the same amount if we set the value of the neutral outcome to zero. Therefore, we examined the effects of κ , instead of β , while setting the value of the neutral outcome to zero. Fig. 3(A) shows the estimated regression coefficients with varying κ . As expected, the outcome value had a monotonic effect on the regression coefficients over the entire reward history. In addition, for the regression coefficients for choice history, the outcome value also had a monotonic effect; the larger the κ , the greater the negative dependence on the choice history was found to be. Fig. 3(B) shows the effects of the learning rate αL on the influence of reward and choice history. The basic effect of the learning rate observed for the F-Q model (αL = αF ; Fig. 1(A)) is also retained for the standard Q-learning model. For the influence of reward in the immediate past, the learning rate has a positive monotonic effect. In contrast, for the reinforcement effect in the far past, the relation is inverted. The larger the learning rate, the greater the decay of the regression coefficient. In contrast to the F-Q model, where the sum of the regression coefficients is independent of the learning rate (Eq. (18)) if the cut-off effect can be neglected, the standard Q-learning model results in a region of decreasing total regression coefficients as a function of the learning rate αL (Fig. 3(B), third panel from top), in addition to an increasing region because of the cut-off effect. This is a result of the unchanging value of the unchosen option in the standard Q-learning model. Because the effect of retaining the value of the unchosen option is enhanced for the case with a lower learning rate, the total impact of reward history becomes smaller as the learning rate increases, which shapes the decreasing part. It is notable that for the case with αL = 1.0 in the standard Q-learning model, even the reward two trials ago has an influence (Fig. 3(B)) because of the effect of retaining the value of the unchosen option. In contrast, in the F-Q model 66 K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 A B Fig. 4. The effects of reinforcement schedules. The Q-learning model parameters were set to αL = 0.4, αF = 0.0, κ = 1.0, and β = 3.0. (A) The effects of the reward probability in the probabilistic learning task. (B) The effects of the withdrawal rate in the competitive foraging task. See the main text for details. with αL = αF = 1.0, only a reward given one trial ago has an influence (Fig. 1(A)). In addition, the learning rate has a non-monotonic effect on the regression coefficients of the choice history (Fig. 3(B), middle). The impact of the choice history has a maximum at approximately αL = 0.5. 4.4. The effects of the reinforcement schedule Our analytical calculation demonstrated that for the special case where the forgetting rate αF equals the learning rate αL , the regression coefficient is determined independent of the task structure, i.e., the reinforcement schedule. For the general case where αL ̸= αF , however, the reinforcement schedule may affect the influence of the previous reward history because the impact of the reward history depends on the number of the same choices after the reward is given, as shown in Eq. (21). To examine this effect, first, we conducted a simulation with varying reward probabilities for the optimal option (pr ) from 0.5 to 0.9 (with reward probability for the non-optimal option being 1 − pr ). Fig. 4(A) shows the regression coefficients obtained by the simulation. The closer to 0.5 that pr was (the more difficult it is to discriminate the optimal choice), the smaller the decay of the regression coefficients, although the effect was weak. This result is explained as follows. When the difference in reward probabilities of two options is small, the difference between two action values tends to be small; thus, the model is likely to switch the choice. Therefore, the number times the same option is repeated becomes smaller, which leads to a smaller decay of the influence of reward history. In the probabilistic learning tasks that we have considered thus far, the reward probability is independent of the subjects’ previous choices. Next, we investigated what happens if the reward is given depending on the previous choice. A typical reinforcement schedule for this situation is a variable interval (VI) schedule. In a VI schedule, a reward is assigned to options stochastically and independently of the subject’s choice, and the reward remains until it is harvested by choosing the option. The probabilistic learning task belongs to a variable ratio (VR) schedule in which a reward is assigned to each option independent of choice, but the reward is immediately withdrawn unless the subject chooses the option and harvests the reward. For the simulation, we adopted the competitive foraging task proposed by Sakai & Fukai (2008) as the reinforcement schedule (see also Katahira, Okanoya, & Okada, 2012). This schedule combines the VI and VR schedules via the introduction of a withdrawal rate that is denoted by µi for option i. With the probability of µi , a reward at option i is removed if the subject does not obtain it. If µa′ = 0, ∀a′ , the task corresponds to the VI schedule. If µa′ = 1, ∀a′ , the task corresponds to the VR schedule. An intermediate region of µa provides mixtures of the VI and VR schedules. Fig. 4(B) shows the regression coefficients for the simulation with varying withdrawal rates (µ, common to both options) and a fixed reward assignment rate of 0.3 for one option and 0.7 for another option. These probabilities remained constant throughout the task (500 trials), i.e., no reversal occurred. The smaller the withdrawal rate (i.e., closer to VI schedule), the smaller the regression coefficient decay of the reward history and the more negative the regression coefficients for the entire choice history. The reason for these effects is as follows. With lower withdrawal rates, the maintenance of the same option following a reward is less likely rewarded in the subsequent trials compared with the other option. Thus, the model tends to switch the option because of the decrease in the action value compared with the case with a smaller or zero withdrawal rate (VR schedule). Therefore, the number of times the same option repeats decreases, and the decay of the influence of reward history becomes slower. 4.5. The effects of the choice-autocorrelation factor We have seen that the standard Q-learning model exhibits a (negative) dependency on choice history, as well as on reward history. However, it is difficult to control the dependence on choice history explicitly by tuning a model parameter. As described above, human choice behavior has a property called choice perseverance, which is a tendency to repeat the same choice as the recent choice (Akaishi et al., 2014; Gershman et al., 2009; Huys et al., 2011). A straightforward way to represent choice perseverance in the Q-learning is simply to add a residual choice-autocorrelation factor to the action values when computing the choice probability, as in Eq. (22). Here, we investigated the effect of the choiceautocorrelation factor on history dependence using a probabilistic learning task (with pr = 0.7). K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 A 67 B Fig. 5. The effects of the choice-autocorrelation factor. (A) The FQ-learning model with varying ϕ (choice autocorrelation weight). The parameters were set to αL = 0.4, αF = 0.4, κ = 1.0, β = 3.0, and τ = 0.4. (B) The standard Q-learning model with varying ϕ (choice autocorrelation weight). The parameters were set to αL = 0.4, αF = 0.0, κ = 1.0, and β = 3.0. The task used for simulation was the same probabilistic learning task for which results are shown in Figs. 1–3. For the F-Q model (αF = αL ), the impact of the choice autocorrelation factor on the regression coefficients can be evaluated analytically (Eq. (25)). Fig. 5(A) confirms the analytical relation: The results of the simulation (solid lines) and theoretical prediction agreed. The choice-autocorrelation factor does not influence the dependence on the reward history (all of the solid lines overlapped each other); instead, the effect of the choice-autocorrelation factor appeared in the regression coefficients for the choice history. On the other hand, for the general case in which αL ̸= αF , the choice-autocorrelation factor can also influence the reward history because it directly affects the number of identical choices that are made after a reward is given, and this effect influences the regression coefficients for the reward history, as discussed above (see Eq. (21)). Fig. 5(B) shows the simulation results for the standard Q-learning model with αF = 0 and illustrates one example of this effect. As ϕ increases in a positive domain, the tendency to repeat the same choice increases and enhances the decay of the influence of the reward history compared to the case in which there was no choice-autocorrelation factor (ϕ = 0). As ϕ decreases in a negative domain, the opposite effect is observed. The residual choiceautocorrelation factor has an additive effect on the regression coefficients for the choice history, bc . Taken together, the effects of the choice autocorrelation factor on the dependence on the history are largely additive and straightforward. For the general case, however, this factor may modulate the dependence on reward history through the property that we have observed in previous results, i.e., that it depends on the number of times that the option is chosen. 5. Discussion RL and linear regression models are valuable tools for the analysis of the psychological/neural processes that underlie choicesequence data on a trial-by-trial basis. However, the relation between the two models has not been described, and these models have been used independently. In the present study, we aimed to clarify the relation between the two approaches using analytical calculations and computer simulations. Here, we summarize our findings and describe their implications. We demonstrate the special condition where the Q-learning and regression models provide an identical prediction. This is the case when the following three conditions hold: (1) the forgetting and learning rates are identical (αL = αF ; F-Q model), (2) the influence of the initial action value can be neglected (e.g., the initial values are zero), and (3) the cutoff effect of the history length (Mr ) included in the regression model can be neglected. When the first condition is not met (i.e., when the forgetting rate differs from the learning rate), as is the case for the standard Q-learning model (αF = 0), a dependence on choice history arises. This dependence on choice history is relatively complex and cannot be completely captured by a conventional regression model. This dependence on choice history is a property of the RL model, which distinguishes it from simple regression models. It should be noted that this apparent dependence on choice history was found only by examining the relationship between regression models and RL models, as we have done in the present study, rather than by simply examining the RL model itself. An apparent dependence on choice history in the general Qlearning model arises from differences in the ‘‘clocks’’ for different options. In the standard Q-learning model (αF = 0), the action value for the unchosen option does not change, i.e., the clock stops at this trial. In contrast, in the F-Q model (αF = αL ), the clock proceeds at the same speed for all options; for the unchosen option, the value decays at the same rate that the option is chosen, but no reward is given. Thus, for the standard Q-learning model, a reward in a more distant past can influence a future choice, depending on how many times the option was chosen. The more general case (αF ̸= αL ) can be understood by this analogy. When αF < αL , the clock for unchosen option moves more slowly than the clock for the chosen option. In contrast, for the case αF > αL , the clock for the unchosen option moves more quickly than the clock for the chosen option; thus, a reward in the more distant past from a frequently chosen option influences a future choice less. If the actual computational process in a decision-maker is similar to that employed in the standard Q-learning model, i.e., the value of the unchosen option remains unchanged, better predictions could be achieved by constructing a regressor of the regression with different clocks for each option. Specifically, such a model should include the variables that represents reward or 68 K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 choice n trials back in trials in which that option was chosen, rather than in actual trials (as in the method discussed in this paper). However, for more general cases (αF ̸= αL , αF ̸= 0), mapping the RL model to the regression model is not straightforward. Hence, which RL model is most appropriate does matter for determining which kind of regression model is most suitable for analyzing a given data set. Ito and Doya (2009) fitted various models to rat choice behavior. Their results demonstrated that the F-Q model (αL = αF ) better fit the rats’ choice behaviors compared with the standard Q-learning model with αF = 0, and the F-Q model had approximately the same prediction performance as the Q-learning model with differential forgetting (DF-Q model; αL and αF were independent free parameters). The F-Q model was ultimately favored because the F-Q model has fewer parameters than the DF-Q model. Together with our discussion in the present study, Ito and Doya’s results may suggest that the simple regression model (with an actual trial count) would suffice to describe choice behavior in rats. We have discussed the similarities and differences between RL models and regression models. Which approach researchers should use depends on the goals and actual computational processes of the decision-maker. The functional forms of the dependence on reward history in the Q-learning model are restricted to exponential decay by the model structure, whereas in regression models, the dependence on reward history can take any form. If the pattern of decay is not exponential, for example, if it is a heavy-tailed, double-exponential form (Corrado et al., 2005), and if the goal is to predict a future choice, a linear regression model may be more appropriate and may provide a better prediction. In contrast, if the actual learning process is similar to that in the general Q-learning model rather than that in the F-Q model, the Q-learning model may provide better predictions. In addition, the Q-learning model may provide a better prediction when the model has a smaller number of parameters than the regression model does. Moreover, as the RL model can represent the value update rule explicitly, one can easily grasp the computational process with this representation. For example, it is not straightforward how the computational process can lead to a double-exponential decay pattern, and several computational models have been generated to attempt to explain this pattern (eg., Saito, Katahira, Okanoya, & Okada, 2014). Most importantly, the variables in the RL model, such as action values and reward prediction errors, can provide a regressor of neural activities that can be used to find the corresponding neural substrates (Daw, 2011; O’Doherty et al., 2007). Furthermore, including a factor that can affect behavior and testing the effects of such a factor, based on data, are often straightforward with RL model-based approaches compared to regression approaches. The RL-model parameters, such as the learning rate, inverse temperature, and outcome values, have been used to characterize several factors of decision making, such as the social attitudes of individuals (Lindström et al., 2014), effects of mental disorders (Kunisato et al., 2012; Yechiam et al., 2005), brain dysfunctions (Yechiam et al., 2005), and effects of emotional outcome (Katahira et al., 2014, 2011, 2015). Model-based analysis is a promising approach to investigating the specific parameters that best characterize the factors that influence learning and behavior. However, it is important to consider which types of differences in properties of behavior, e.g., dependence on past experience, choice bias, or choice randomness, yield differences in the model parameters. The present study provides guidelines for relating the effects identified in model-based analysis to the properties of actual behavior. Below, we summarize how the RL-model parameters are related to the history dependence of choice; a portion of these results was obtained in the present study. First, the learning rate αL largely controls how the weights for past outcomes are balanced, i.e., how much the model weighs more recent outcomes compared to outcomes in the more distant past. In the F-Q model (αL = αF ), the learning rate does not influence the total weight (the sum of the regression coefficients for reward history). However, we demonstrated that if the learning rate and the forgetting rate differ, then the total weight can be a decreasing function of the learning rate. This finding implies that increasing αL does not necessarily lead to an increase in the cumulative effect of the recent reward history. Therefore, the value of the learning rate should be interpreted with caution. The inverse temperature β and the outcome value κ had essentially the same effect on the history dependence unless the outcome values varied for different outcomes. These parameters uniformly and multiplicatively control the weights for past events. Thus, the summed influence of the past reward history is a monotonically increasing function of these parameters. The residual choice-autocorrelation factor Ci (t ) has an additive effect on the dependence on choice history. In the general case, this factor may modulate the dependence on reward history. One limitation of the present study is that it only concerned learning tasks without state transition dynamics that can be formulated as Markov decision processes. Although much research in psychology and neuroscience has been conducted using simplified Q-learning models without state variables, as discussed in the Introduction, several studies using RL model-based analyses have examined RL models that incorporate state variables (Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Gläscher, Daw, Dayan, & O’Doherty, 2010). In addition, we have focused on a specific algorithm of RL, i.e., Q-learning. Other algorithms for RL have been used for model-based analyses of choice behavior, e.g., actor-critic algorithm (O’Doherty et al., 2004). Although the scope of the present study is limited, the basic results found here can be applied to the more general case. Other algorithms that incorporate learning state transitions (Daw et al., 2011; Gläscher et al., 2010) essentially employ a sequential update rule, such as a delta rule (as in Q-learning); thus, these algorithms likely produce exponential decay in history dependence. The main results of the present study are the findings that if all of the variables decay at the same speed, then the history dependence can be represented using an appropriate regression model, but if the speeds differ or if updating is stopped when the variable is irrelevant to the current experience, then the history dependency would be more complicated. Future studies are needed to investigate the history dependence of choices in more general RL models, including models that include state variables and models with different learning algorithms. Acknowledgment This work was supported in part by the Grants-in-Aid for Scientific Research (KAKENHI) grants 24700238 and 26118506. References Akaishi, R., Umeda, K., Nagase, A., & Sakai, K. (2014). Autonomous mechanism of internal choice estimate underlies decision inertia. Neuron, 81(1), 195–206. Barraclough, D. J., Conroy, M. L., & Lee, D. (2004). Prefrontal cortex and decision making in a mixed-strategy game. Nature Neuroscience, 7(4), 404–410. Corrado, G., & Doya, K. (2007). Understanding neural coding through the modelbased analysis of decision making. Journal of Neuroscience, 27(31), 8178. Corrado, G., Sugrue, L. P., Seung, H. S., & Newsome, W. T. (2005). Linear-nonlinearPoisson models of primate choice dynamics. Journal of the Experimental Analysis of Behavior, 84(3), 581–617. Daw, N. (2011). Trial-by-trial data analysis using computational models. In Decision making, affect, and learning: attention and performance XXIII 23, 1. Daw, N., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215. http://dx.doi.org/10.1016/j.neuron.2011.02.027. Gershman, S. J., Pesaran, B., & Daw, N. D. (2009). Human reinforcement learning subdivides structured action spaces by learning effector-specific values. Journal of Neuroscience, 29(43), 13524–13531. http://dx.doi.org/10.1523/JNEUROSCI. 2469-09.2009. Gläscher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying modelbased and model-free reinforcement learning. Neuron, 66(4), 585–595. http://dx.doi.org/10.1016/j.neuron.2010.04.016. Huys, Q. J., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011). Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Computational Biology, 7(4), e1002028. K. Katahira / Journal of Mathematical Psychology 66 (2015) 59–69 Ito, M., & Doya, K. (2009). Validation of decision-making models and analysis of decision variables in the rat basal ganglia. Journal of Neuroscience, 29(31), 9861. Katahira, K., Fujimura, T., Matsuda, Y.-T., Okanoya, K., & Okada, M. (2014). Individual differences in heart rate variability are associated with the avoidance of negative emotional events. Biological Psychology, 103, 322–331. http://dx.doi.org/10.1016/j.biopsycho.2014.10.007. Katahira, K., Fujimura, T., Okanoya, K., & Okada, M. (2011). Decision-making based on emotional images. Frontiers in Psychology, 2, 311. Katahira, K., Matsuda, Y.-T., Fujimura, T., Ueno, K., Asamizuya, T., Suzuki, C., et al. (2015). Neural basis of decision-making guided by emotional outcomes. Journal of Neurophysiology, in press. Katahira, K., Okanoya, K., & Okada, M. (2012). Statistical mechanics of rewardmodulated learning in decision-making networks. Neural Computation, 24(5), 1230–1270. Kovach, C. K., Daw, N. D., Rudrauf, D., Tranel, D., O’Doherty, J. P., & Adolphs, R. (2012). Anterior prefrontal cortex contributes to action selection through tracking of recent reward trends. Journal of Neuroscience, 32(25), 8434–8442. http://dx.doi.org/10.1523/JNEUROSCI.5468-11.2012. Kunisato, Y., Okamoto, Y., Ueda, K., Onoda, K., Okada, G., Yoshimura, S., et al. (2012). Effects of depression on reward-based decision making and variability of action in probabilistic learning. Journal of Behavior Therapy and Experimental Psychiatry, 43(4), 1088–1094. http://dx.doi.org/10.1016/j.jbtep.2012.05.007. Lau, B., & Glimcher, P. W. (2005). Dynamic response-by-response models of matching behavior in rhesus monkeys. Journal of the Experimental Analysis of Behavior, 84(3), 555–579. Lindström, B., Selbing, I., Molapour, T., & Olsson, A. (2014). Racial bias shapes social reinforcement learning. Psychological Science, 25(3), 711–719. Niv, Y. (2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669), 452–454. 69 O’Doherty, J., Hampton, A., & Kim, H. (2007). Model-based fmri and its application to reward learning and decision making. Annals of the New York Academy of sciences, 1104(1), 35–53. Saito, H., Katahira, K., Okanoya, K., & Okada, M. (2014). Bayesian deterministic decision making: a normative account of the operant matching law and heavy-tailed reward history dependency of choices. Frontiers in Computational Neuroscience, 8. Sakai, Y., & Fukai, T. (2008). The actor-critic learning is behind the matching law: matching versus optimal behaviors. Neural Computation, 20(1), 227–251. Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005). Representation of actionspecific reward values in the striatum. Science, 310(5752), 1337. Schultz, W. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. http://dx.doi.org/10.1126/science.275.5306.1593. Seo, H., Barraclough, D. J., & Lee, D. (2009). Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. Journal of Neuroscience, 29(22), 7278–7289. http://dx.doi.org/10.1523/JNEUROSCI.1479-09.2009. Seo, H., & Lee, D. (2009). Behavioral and neural changes after gains and losses of conditioned reinforcers. Journal of Neuroscience, 29(11), 3627–3641. http://dx.doi.org/10.1523/JNEUROSCI.4726-08.2009. Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012). Serotonin selectively modulates reward value in human decision-making. Journal of Neuroscience, 32(17), 5833–5842. http://dx.doi.org/10.1523/JNEUROSCI.005312.2012. Sugrue, L. P., Corrado, G., & Newsome, W. T. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304(5678), 1782–1787. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge, MA, USA: MIT Press. Watkins, C., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3), 279–292. Yechiam, E., Busemeyer, J., Stout, J., & Bechara, A. (2005). Using cognitive models to map relations between neuropsychological disorders and human decisionmaking deficits. Psychological Science, 16(12), 973–978. JPII 8 (1) (2019) 75-80 Jurnal Pendidikan IPA Indonesia http://journal.unnes.ac.id/index.php/jpii ASSESSMENT OF SEVENTH GRADE STUDENTS’ CAPACITY OF LOGICAL THINKING M. Fadiana*1, S. M. Amin2, A. Lukito3, A. Wardhono4, S. Aishah5 Department of Mathematic Education, Faculty of Teacher Training and Education, Universitas PGRI Ronggolawe Tuban 2,3 Department of Mathematic, Faculty of Math and Science, Universitas Negeri Surabaya 5 Department of Defence Science, Faculty of Defence Science and Technology, National Defence University of Malaysia 1,4 DOI: 10.15294/jpii.v0i0.11644 Accepted: October 2nd, 2018. Approved: March 25th, 2018. Published: March 28th, 2019 ABSTRACT The objective of this research was to quantify the logical thinking capacity of seventh grade students in Tuban, East Java, Indonesia. This research was conducting using the quantitative descriptive method and 119 students in the seventh grade of secondary school in Tuban during the Academic Year 2016/2017. The data was collected by using Logical Thinking Test (LTT) which comprises of six different kind of reasonings; namely conservational reasoning, proportional reasoning, controlling variables, combinatorial reasoning, probabilistic reasoning, and correlational reasoning (source). Based on LTT, scores are categorized into three levels; concrete operational levels, transitional levels, and formal operational levels (source). The results of the research display that (88.21%) of the seventh grade are classified at the concrete operational level, (10.08%) at the transitional level, and (1.68%) at the formal operational level. After conducting this research, teachers are now able to design teaching tactics and have a better understanding of secondary school student’s cognitive development and behavior. © 2019 Science Education Study Program FMIPA UNNES Semarang Keywords: logical thinking, concrete operational, formal operational INTRODUCTION Logistical thinking is a though process using logic, rationality and reason (source). Logic is defined as the discipline that examines the assembly of information and differentiates it between right and wrong reasoning (source). Logic can also be identified as the tool of correct thinking, the key to the processes of mental reservation and complex problem-solving (source). This means it have the capability to solve problems by using mental operations or one’s ability to range *Correspondence Address E-mail: [email protected] principles or rules by making confidentgeneralizations or ideas (Yaman, 2005; Fero et al., 2009; Şengül & Üner, 2010). Logical thinking is one of the ways used in acquiring advanced mental activities (source). This ability is an application level activity that depends on the knowledge and understanding levels of the objective’s cognitive stage (source). Also, Logical thinking is used in evaluating ideas, experiences, and information. Our logic produces results relate to the topic we are interested in, and then it converts these to memory. Piaget, a Swiss psychologist who is known for this work on child’s development, defines lo- 76 M. Fadiana, S. M. Amin, A. Lukito, S. Aishah, A. Wardhono / JPII 8 (1) (2019) 75-80 gical thinking as an ability that is observed in the concrete and abstract operations stage (source). When students are in the concrete operations stage, they can use logical thinking abilities in solving concrete problems (source). In the abstract operations stage, students obtain the level of adults in terms of rational thinking. According to Demirel & Coşkun (2010) and Lee & Bednarz (2012)’s study, logical thinking includes effective use of numbers, finding scientific solutions to problems, differentiating among concepts, classification, generalizing, calculations, and providing hypotheses. Roadrangka states three developmental stages by utilizing the logical thinking levels including concrete, transitional, and formal (Roadrangka, 1995). It can also be referred to as providing us with information about an individual’s cognitive development level (source). The important question to ask is “Why is logical thinking important in science learning?” After several research studies, data shows that there is a significant relationship between abstract thinking and scientific process skills and success in chemistry (Oloyede, 2012). Students using abstract thinking tend to be more successful compared to those who do not because lowlevel reasoning will bring low-level performance (Oloyede, 2012; Piaget & Inhelder 2013). Science requires the skills of collecting and analyzing data to solve problems, to formulate hypotheses, to control variables and to define them operationally (source). Such processes require a high level of logical thinking ability. Proportional reasoning is important in the quantitative aspect of chemistry, specially to understand the origin and use of many useful relationships in chemistry, such as the development and interpretation of tabulation and graph data (Ruiz & Lupiáñez, 2009). In addition, correlational reasoning plays an important role in the formulation of hypotheses and data interpretations that needed to consider relationships between variables. The controlling variable has an important role in planning, implementation, and interpretation (source). Before designing the learning process in the classroom, the teacher should know the level of the students’ logical thinking abilities (Gómez, 2007). It is necessary for teachers to be able to design a learning strategy in accordance with the level of logical thinking ability of their students. As Othman et al. (2010) stated, many people fail to realize that logical thinking is among the most important factors in determining the qualifications of students in learning programs (Othman et al., 2010). Program development specialist have an important role to play by making special efforts to have a better understanding of the world for children and offer academic experience based on the children’s curiosity and demand (Simatwa, 2010). Assessment of the logical thinking capability can also be used as a basis for measuring the mastery of science materials (Fah, 2009). The mastery of science materials can be predicted based on logical thinking ability. This opinion is in line with the results of Oliva’s research (2003), students with high logical thinking ability can change their alternative conceptions more easily. Moreover, logical reasoning of learners makes a thinking style impact on the ability to solve Physics problems (Bancong, 2013; Etzler & Madden, 2014; Rakhmawan & Vitasari, 2016). However, the researcher often encountersthat teachers rarely measure students’ logical thinking skills before designing learning strategies to be undertaken. Therefore, the ability of logical thinking level of seventh grade students is examined. This study is very important in relations to defining logical thinking levels of students. This is useful for education experts and program development specialists because it affects various characteristics such as; the acquisition of scientific concepts, three-dimensional thinking, and scientific process skills (source). The researcher motivation for the study was to inspire and assist teachers in design learning strategies in the classroom. The researcher aims to answer the following research question: What is the fundamental difference found between the formal operational students, transitional students, and concrete operational students? METHODS This study uses the quantitative approach with non-experimental research designs in the form of surveys (Creswell, 2009). This survey method is used to obtain data in certain places where the datataken consists of natural data without treatment as well as experiments (Sugiyono, 2008).The sample that was used in this study was of 119 students (61 male and 58 female students) from a private junior high school in Tuban, East Java, Indonesia during the academic year 2016/2017. M. Fadiana, S. M. Amin, A. Lukito, S. Aishah, A. Wardhono / JPII 8 (1) (2019) 75-80 The instrument that was used in this research is Logical Thinking Test (LTT) which was revised from the Group Assessment of Logical Thinking (GALT). GALT was created to measure six different kind of reasonings; namely conservational reasoning, proportional reasoning, controlling variables, combinatorial reasoning, probabilistic reasoning, and correlational reasoning (Roadrangka et al., 1983). The test was formed to uses multiple answers while an image is inserted for each item to help visualize the problem (source). In order to acquire a valid and reliable research instrument, the researcher tested cogency and consistency of LTT. LTT has been validated by experts who are English Linguists, psychologists, science teachers, ad mathematics teachers. Efforts were made to confirm the content face validity of the adapted and translated version of the instrument. When the results were translated into Indonesian (the national language called Bahasa Indonesia) so that the respondents could then understand the items and choose the best results. LTT also administered a reliability test. The coefficient of reliability (KR-20) of the LTT is 0.83. Thus, the selection of question originating from a valid and reliable instrument resulted in the measurement of students’ logical thinking ability to be valid and reliable. The research sample was given LTT to be completed in 50 minutes. Students answer LTT on the answer sheet provided. Each type of operations is represented by each of the two questions, in which five types of reasonings in the form of multiple choices are accompanied by reasons. As for the type of combinatorial reasoning, participants must explain each possible answer. The answer for the item of LTT number 1-10 is correct if both the answer and reason are correct. For items number 11 and 12 (combinatorial reasoning), the students are required to write down the answers. The answer in the minimum score is 0. Next, the score is classified as follows; scores 0-4 are ground in concrete operational levels, 5-7 scores are grouped in transitional levels, and 8-12 are formal operational levels. 77 vels if they score 8-12. Based on the LTT that has been completed by the students, the writers obtain data, as shown in Table 1 Table 1. Logical Thinking Ability Levels of Seventh Grade Student at a Private Junior High School In Tuban, East Java, Indonesia Level of Logical Thinking Number of Student Percentage Formal Operational 2 1.68 % Transitional 12 10.08 % Concrete Operational 105 88.24 % Based on Table 1, (1.68%) of the seventh students were at theformal operational level, (10.08%) were at the transitional level and the remaining (88.21%) were at theconcrete operational level. The results of this study are consistent with the results of research from previous researchers. Bitner (1991) found thatseventh Grade students (n = 156), (5%) were at formal operational levels, 33% were at transitional levels, and 62% were at concrete operational levels. On the other hand, Promo and Fahey (1982) reported the results of their research, indicatingthat (3.5%) of seventhgrade students are at the formal operational levels. Similar results were also distint in the literature (Biber et al., 2013; Bulut et al., 2009; Şenlik, Balkan,& Aycan, 2011). Based on the measurement of students’ logical thinking ability classified according to the gender division of male and female, the results are presented as shown in Figure 1. RESULT AND DISCUSSION After the students have completed the LTT, the answer sheets are scored according to the scoring guidelines. Furthermore, the scores are derived from the three levels. If students attain a score of 0-4, then they are grouped into the category of concrete operational levels. If students score 5-7, they are grouped into transitional levels, and students are grouped into formal operational le- Figure 1. Logical Thinking Ability When Viewed Based on the Gender of Male and Female of Seventh Grade Student at a Private Junior High School in Tuban, East Java, Indonesia Based on Figure 1, the number of students who are at the formal operational levelsconsist 2 78 M. Fadiana, S. M. Amin, A. Lukito, S. Aishah, A. Wardhono / JPII 8 (1) (2019) 75-80 male students only. While at the transitional level there are 9 male and 3 female students. At the oncrete operational levels there are 50 male students and 55 female students. Based on the results of this study, the proportion of logical thinking ability between male and female students is similar. Researchers Yaman, Fah, Kincal and Tuna indicated that based on the gender variable, that was no major change in the mean for this variable (Yaman, 2005; Fah, 2009; Kıncal et al., 2010; Tuna et al., 2013). As an example, Fah (2009) in his research to investigate the logical thinking ability of Sabah Malaysian students whose population is 16 years old, (97.2%) of male respondents and (98.7%) of female respondents are classified as the transitional operational stage. This is consistent with the results of researcher Kincal, who declared that logical thinking when it came to gender variable that there was no major difference (Kincal et al., 2010). However, when it came to the variables of the different type of school such as; academic success, socio-economic background and socio-cultural background that there was a major different between the scores (Kincal et al., 2010). After all this research and data, it is safe to conclude that the gender variable provides a feeble and unreliable measure for any decision of logical thinking. However, when viewed per item matter, male students are superior to female students when it comes tocorrelational reasoning and probabilistic reasoning. By contrast, female students are superior to male students in this type of combinatorial reasoning.The results of the LTT analysis by the research subjects for each type of reasoning are presented in Table 2. Table 2. Logical Thinking Ability of Seventh Grade Students at a Private Junior High School In Tuban, East Java, Indonesia for Each Type of Reasoning Operation No. 1 2 3 4 Type of Operation Conservation Item No. Theme Number of Students with Correct Answers Percentage 1 Piece of Clay 38 31,9 % 2 Metal Weights 21 17, 6 % Correlational Reasoning 3 Glass Size 15 12, 6 % 4 Scale 22 18, 5 % Proportional Reasoning 5 Pendulum Length 20 16, 8 % 6 Ball 25 21 % Control Variable 7 Squares and Diamonds #1 13 10, 9 % 8 Squares and Diamonds #2 20 16, 8 % 5 Probabilistic Reasoning 9 The Mice 8 6,7 % 10 The Fish 6 5% 6 Combinatorial Reasoning 11 The Dance 86 7, 2 % 12 The Shopping Center 45 37,8 % Based on Table 2, the result indicates that combinatorial reasoning attains the highest average score, while reasoning reason has the lowest average score. Fah showed similar results, namely that the lowest are probability reasoning scores, but the highest scores can be found in the category of combinatorial reasoning (Fah, 2009). Yenilmez et al., (2005) and Sezen & Bülbül, (2011) scored highest in the reasoning of variable control, while the lowest was found in correlational reasoning (Yenilmez, 2005).This difference shows that the development of logical thinking ability of each student isdifferent, and obviously influenced by the environment that shaped it. There is a fundamental difference found in the results of the study between theformal and operational level of the students and the level of non-formal operations. In the case of proportional reasoning, concrete and transitional level operational students tend to use intuitive and additive reasoning rather than using rational reasoning in solving problems. In the problem of controlling variables, students in the categories oftheconcrete operational levels and transitional levels do not show an understanding of the relationship between manipulation and control. In solving probabilisticproblems, students at the concrete operational level and transitional level focus only on M. Fadiana, S. M. Amin, A. Lukito, S. Aishah, A. Wardhono / JPII 8 (1) (2019) 75-80 one or two dimensions of the problem (i.e., geometric rhombic shape and number of diamonds). They are unable to observe the characteristics of the object and understand the relationship between these characteristics on the problem of correlational reasoning. Students who are at the non-formal operational level (concrete and transitional operational level) have not been able to show the pattern and cannot solve all combinations in the problem of combinatorial logic. Roadrangka stated there is a correlation between formal operational reasoning capabilities and the student’s achievement in biology, physics, and chemistry (Roadrangka, 1995). At the formal operational stage, students scored higher in science, material science and science tests which was different for those who were a definite operational stage and understudies at the transitional operational stage (Roadrangka, 1995). This was declaring that students were unable to expand this understanding of theoretical ideas. Therefore, students who are successful in science would be certain by using different modes of formal operational reasoning (Tsaparlis, 2005; Tai et al., 2005; Lewis & Lewis, 2007; Fabelo et al., 2011). Lewis and Lewis highlighted the important needs to include a focus on the development of formal beliefs as well as content review in the way to help at-risk students in general chemistry classes (Lewis & Lewis, 2007). Therefore, in science and mathematics learning, bridges are needed to reduce the gap between formal operational and non-formal operational stages. Science and mathematics learning can make use of concrete learning media in order to make the abstract concept easier for students to understand. Interactive multimedia can encouragesuccess anda more advance thinking skill for science students today (Melida, 2014; Alimah, 2012; Hartini et al., 2017). In addition to using the media, teachers could apply cooperative learning methods. Cooperative learning methods will be improving the students’ logical thinking levels thus improving their performances (Othman et.al., 2010; Eskandar et al., 2013; Glen, 2013). CONCLUSION Assessment of students’ logical thinking at a private junior high school in Tuban, East Java Indonesia taken during Academic Year of 2016/2017 which consisted of 61 male and 58 female students had the following outcome; (1. 68%)are at formal operational levels, (10.08%) are at the transitional levels and (88.24%) are at concrete operational levels. The proportion of lo- 79 gical thinking among male and female students is similar. However, there is a difference between formal operational students and non-operational formal students (transitional students and concrete operational students) when it comes to relative thinking, control factors, probabilistic thinking, correlational thinking, and combinatorial thinking. It is important to recognize that since majority of the students that participated in this research are in the concrete operational period, there may have been influence from cultural variables, educational framework, and reading behaviors. Also, keeping in mind the grade level impact of coherent reasoning which could be from instructions and instructive dimension are in this manner likewise vital in intelligent reasoning. REFERENCES Alimah, S. (2012). Pengembangan Multimedia Pembelajaran Embriogenesis Hewan untuk Mengoptimalkan Pemahaman Kognitif Mahasiswa. Jurnal Pendidikan IPA Indonesia, 1(2). Bancong, H. (2013). Profil Penalaran Logis Berdasarkan Gaya Berpikir Dalam Memecahkan Masalah Fisika Peserta Didik. Jurnal Pendidikan IPA Indonesia, 2(2), 195-202. Biber, A. C., Tuna, A., & Incikabi, L. (2013). An Investigation of Critical Thinking Dispositions of Mathematics Teacher Candidates. Journal of International Educational Research, 9(3), 257-266. Bulut, S., Ertem, G., & Sevil, Ü. (2009). Examination of the Critical Thinking Levels of Nursing Students. Dokuz Eylül University Electronic Journal of School of Nursing , 2 (2), 27-38. Creswell, J. W. (2009). Mapping the Field of Mixed Methods Research. Demirel, M., & Coşkun, Y. D. (2010). Case Study on Interdisciplinary Teaching Approach Supported by Project Based Learning. The International Journal of Research in Teacher Education, 2(3), 28-53. Etzler, F. M., & Madden, M. (2014). The Test of Logical Thinking as a Predictor of First-Year Pharmacy Students’ Performance in Required First-Year Courses. American Journal of Pharmaceutical Education, 78(6), 121. Eskandar, F. A., Bayrami, M., Vahedi, S., & Ansar, V. A. A. (2013). The Effect of Instructional Analogies in Interaction with Logical Thinking Ability on Achievement and Attitude Toward Chemistry. Chemistry Education Research and Practice, 14(4), 566-575. Fabelo, T., Thompson, M. D., Plotkin, M., Carmichael, D., Marchbanks, M. P., & Booth, E. A. (2011). Breaking Schools’ Rules: A Statewide Study of How School Discipline Relates to Students’ Success and Juvenile Justice Involvement. New York: Council of State Governments Justice Center. 80 M. Fadiana, S. M. Amin, A. Lukito, S. Aishah, A. Wardhono / JPII 8 (1) (2019) 75-80 Fah, L. Y. (2009). Logical Thinking Abilities among Form 4 Students in the Interior Division of Sabah, Malaysia. Journal of Science and Mathematics Education in Southeast Asia, 32(2), 161-187. Fero, L. J., Witsberger, C. M., Wesmiller, S. W., Zullo, T. G., & Hoffman, L. A. (2009). Critical Thinking Ability of New Graduate And Experienced Nurses. Journal of Advanced Nursing, 65(1), 139148. Glen, N. J. (2013). Helping Pre-service Elementary Teachers Write Scientific Arguments and Understand Their Connection to the Common Core State Standards. Double Helix, 1. Gómez, P. (2007). Learning Processes in the Initial Training of High School Mathematics Teachers. Hartini, S., Misbah, M., Dewantara, D., Oktovian, R. A., & Aisyah, N. (2017). Developing Learning Media Using Online Prezi into Materials about Optical Equipments. Jurnal Pendidikan IPA Indonesia, 6(2), 313-317. Kıncal, R. Y., & Yazgan, A. D. (2010). Investigating the Formal Operational Thinking Skills of 7th and 8th Grade Primary School Students According to Some Variables. Elementary Education Online, 9(2), 723-733. Lee, J., & Bednarz, R. (2012). Components of Spatial Thinking: Evidence from a Spatial Thinking Ability Test. Journal of Geography, 111(1), 15-26. Lewis, S. E., & Lewis, J. E. (2007). Predicting At-Risk Students in General Chemistry: Comparing Formal Thought to a General Achievement Measure. Chemistry Education Research and Practice, 8(1), 32-51. Melida, D. (2014). Pengaruh Media Prezi the Zooming Presentations terhadap Hasil Belajar Fisika Siswa Kelas Xi Sma N 12 Padang. Pillar of Physics Education, 4(2). Oliva, J. M. (2003). The Structural Coherence of Students’ Conceptions in Mechanics and Conceptual Change. International Journal of Science Education, 25(5), 539-561. Oloyede, O. I., & Adeoye, F. A. (2012). The Relationship between Acquisition of Science Process Skills, Formal Reasoning Ability and Chemistry Achievement. International Journal of African & African-American Studies, 8(1), 1-4. Othman, M., Hussain, F. M., & Nikman, K. (2010). Enhancing Logical Thinking among Computer Science Students through Cooperative Learning. Gading Journal for the Social Sciences, 14(01). Piaget, J., & Inhelder, B. (2013). The Growth of Logical Thinking from Childhood to Adolescence: An Essay on the Construction of Formal Operational Structures. Routledge. Rakhmawan, A., & Vitasari, M. (2016). Kemampuan Berpikir Logis sebagai Prediktor Keberhasilan Mahasiswa dalam Perkuliahan Kimia Dasar. Jurnal Penelitian dan Pembelajaran IPA, 2(1), 99-109. Roadrangka, V., Yeany, R. H., & Padilla, M. J. (1983, April). The Construction of a Group Assess- ment of Logical Thinking (GALT). In 56th Annual Meeting of the National Association for Research in Science Teaching. Dallas, Texas, April (pp. 5-8). Roadrangka, V. (1995). Formal Operational Reasoning Ability, Cognitive Style, and Achievement in Biology, Physics, and Chemistry Concepts of Form 4 students in Penang, Malaysia. SEAMEO Regional Centre for Education in Science and Mathematics, Penang. Ruiz, EF, & Lupiáñez, JL (2009). Detection of Psychopedagogical Obstacles in the Teaching and Learning of the Topics of Reason and Proportion in Sixth Grade Students of Primary Education. Electronic Journal of Research in Educational Psychology , 7(17). Şengül, S., & Üner, İ. (2010). What is the Impact of the Teaching “Algebraic Expressions and Equations” Topic with Concept Cartoons on the Students’ Logical Thinking Abilities?. Procedia-Social and Behavioral Sciences, 2(2), 54415445. Şenlik, NZ, Balkan, Ö., & Aycan, Ş. (2011). Critical Thinking Skills of Prospective Teachers: Muğla University Example. Celal Bayar University Journal of Science , 7(1), 67-76. Sezen, N., & Bülbül, A. (2011). A Scale on Logical Thinking Abilities. Procedia-Social and Behavioral Sciences, 15, 2476-2480. Simatwa, E. M. (2010). Piagets Theory of Intellectual Development and Its Implication for Instructional Management at Pre-Secondary School Level. Educational Research and Reviews, 5(7), 366-371. Sugiyono. (2008). Metode Penelitian Pendidikan: (Pendekatan Kuantitatif, Kualitatif dan R & D). Alfabeta. Tai, R. H., Sadler, P. M., & Loehr, J. F. (2005). Factors Influencing Success in Introductory College Chemistry. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 42(9), 987-1012. Tsaparlis, G. (2005). Non‐Algorithmic Quantitative Problem Solving in University Physical Chemistry: A Correlation Study of the Role of Selective Cognitive Factors. Research in Science & Technological Education, 23(2), 125-148. Tuna, A., Biber, A. Ç., & İncikapı, L. (2013). An Analysis of Mathematics Teacher Candidates’ Logical Thinking Levels: Case of Turkey. Journal of Educational Instructional Studies, 3(1), 83-91. Yaman, S. (2005). Effectiveness on Development of Logical Thinking Skills of Problem Based Learning Skills in Science Teaching. Journal of Turkish Science Education, 2(1), 31-33. Yenilmez, A., Sungur, S., & Tekkaya, C. (2005). Investigating Students’ Logical Thinking Abilities. Hacettepe University Faculty of Education Journal , 28(28), 219-225. International Journal on Emerging Mathematics Education (IJEME) Vol. 1, No. 2, September 2017, pp. 177-184 P-ISSN: 2549-4996, E-ISSN: 2548-5806, DOI: http://dx.doi.org/10.12928/ijeme.v1i2.5783 Creative Thinking Process based on Wallas Model in Solving Mathematics Problem 1Hevy Risqi Maharani, 2Sukestiyarno, 2Budi Waluya 1Mathematics Education Department, Sultan Agung Islamic University Education Department, Semarang State University e-mail: [email protected] 2Mathematics Abstrak Suatu hal yang sulit bagi guru SMP adalah memposisikan dan mengembangkan anak didiknya yang masih berada pada masa transisi dalam berpikir kreatif. Penelitian ini bertujuan untuk mengevaluasi proses berpikir kreatif siswa berdasarkan model Wallas. Penelitian ini ada penelitian deskriptif kualitatif menggunakan data triangulasi. Subjek dikategorikan pada kemampuan tinggi, sedang, dan rendah setelah diberikan tes kemampuan berpikir kreatif. Objek pada penelitian ini adalah eksistensi bagaimana siswa SMP dalam menyelesaikan masalah matematika. Data dianalisis melalui klasifikasi, representasi, dan kesimpulan. Hasil penelitian menunjukkan 1) terdapat 23,33% siswa tidak tuntas dan hanya mencapai tahap persiapan, disebut kelompok kategori rendah; 2) terdapat 60% siswa mencapai tahap iluminasi meskipun untuk sampai pada tahap ini siswa memerlukan waktu lama, disebut kelompok kategori sedang; dan 3) 16,67% siswa telah tuntas sampai tahap verifikasi, disebut kelompok kategori tinggi. Bagi siswa dengan kategori kemampuan rendah dan sedang masih membutuhkan pendampingan saat mengalami hambatan pada proses berpikir kreatifnya, sedangkan siswa kategori kemampuan tinggi membutuhkan materi pengayaan. Kata kunci: proses berpikir kreatif, masalah matematika, Model Wallas Abstract Developing student’s creative thiking is difficult for teacher, especially when they are still in transtition. This study aims to evaluate students’ creative thinking process based on the model of Wallas. This is a descriptive and qualitative research where data triangulation is employed. Subjects are categorized into upper, middle, and low category after doing creative thinking ability test. The object of the study is the existence of how junior high school’s students solving mathematics problems. Data were analyzed through classification, data representation, and conclusions. The results showed 1) 23,33% of students only reached preparation stage, called low category, 2) 60% of students reached illumination stage though students take a long time, called middle category, and 3) 16,67% of students have completed up to verification stage, called upper category. For students in low and middle category, they still need assistance when experiencing obstacles in the creative thinking process, while the upper category students need enrichment materials. Keywords: creative thinking process, mathematics problem, Wallas model How to Cite: Maharani, H.R., Sukestiyarno, & Waluya, B. (2017). Creative thinking process based on wallas model in solving mathematics problem. International Journal on Emerging Mathematics Education, 1(2), 177-184. http://dx.doi.org/10.12928/ijeme.v1i2.5783 INTRODUCTION The development of creative thinking is now expected to be the focus of mathematics education in which students are given the freedom to try to give original or new possible solutions from themselves (Kwon, Park, & Park, 2006). It means, learning mathematics should avoid the use of traditional learning methods that leads to convergent thinking in which students only remember mathematical theorems and Received February 7, 2017; Revised August 22, 2017; Accepted August 29, 2017 178 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 rules to solve problems. In reality, what often happens in mathematics learning is students are given closed problems to be solved. Students are not taught to use divergent thinking and reasoning which is important for them to understand the context so that they give a positive response and actively participate in the learning process. Students who have a creative thinking not only uses mathematical knowledge they have acquired during the learning in solving the problem, but can use new strategies and unusual in solving their problems (Wessels, 2014). Pehkonen (1997) give four reasons why it is important to teach problem solving in relation to creative thinking. First, problem solving can develop cognitive skills. Second, problem solving encourages creativity. Third, problem solving is a part of the process to apply mathematics. And the last, problem solving encourages students to learn mathematics. In Bloom taxonomy, there are two dimensional frameworks: knowledge and cognitive process (Bloom, 1956; Krathwohl, 2002; Huitt, 2011; Bonaci, Mustata & Ienciu, 2013; Munzenmaler & Rubin, 2013; IACBE, 2014). Structure of the knowledge dimension includes factual, conceptual, procedural, and metacognitive knowledge. Structure of the cognitive process dimension includes remember, understand, apply, analyze, evaluate, and create. Create is the highest dimension on cognitive process. Students must have creative thinking ability to get it. Highlighting the sub areas of mathematics, especially geometry, the material consists of concepts that students need more attention to understand them. For instance, junior high school students need transition of thinking process from the concrete to the abstract. The study of Maharani & Sukestiyarno (2015) showed that the average student’s mathematics creative thinking ability on geometry were categorized in almost not creative. There were no students in creative or very creative category. Students in solving the problems still use one particular way that has been taught by their teacher. they were not familiar using variety of ways in solving a problem. The solution given by students were not categorized in a new or unusual idea. Very rare students gave the answer in any variety of ways. Creative thinking ability on the concept of geometry in junior high school cannot be separated from problem solving ability. In defining the relationship between creative thinking and problem solving, it is important to determine what makes creative of creative problem solving. Therefore, it is important to do an investigation of the creative thinking process (Aldous, 2007). One model of the creative thinking process is presented by Wallas (2014). The model consists of four distinct stages: preparation, incubation, illumination, and verification. In the preparation stage we collect a wide range of knowledge, share this knowledge into rules of logic in the realm of the investigation and adopted to define a given problem. During the incubation stage, various ideas freely grouped and rearranged without students directly working on the problem. This stage requires a few seconds, minutes, or hours depending on the difficulty of the problems encountered. Usually when the solution is found, the illumination stage has arrived which is often called as the ‘aha’ experience. At the end, the obtained solution need to be checked, developed, and refined in the verification stage and elaborated to ensure the solutions is understandable. If the verification stage showed a solution to be not feasible then there may be a return to the beginning of creative process. Although preparation and verification stages are included in the conscious activity, incubation and illumination stages are in unconscious activity. In school, students have differences from the other students in many various domains. They have different levels of motivation, background, attitude towards teaching IJEME, Vol. 1, No. 2, September 2017, 177-184 IJEME ISSN: 2549-4996 179 and learning, as well as different responses to the classroom environment and certain instructional practices (Potur & Barkul, 2009; Runisah, Herman, & Dahlan, 2017). As a result, students have differences in creative thinking process. The purpose of this study is to evaluate student’s creative thinking process in solving mathematics problems. If the teachers know the process of creative thinking, the teacher can give the specific assistance to students and can select model or method that appropriate for students. The framework of this study is junior high school students who are in a transition of creative thinking process. In Sari & Sukestiyarno (2014), Widiastuti & Sukestiyarno (2014), and Wulandari & Sukestiyarno (2015), they suggest that in order to achieve the goal of students such as the mastery of cognitive ability in creative thinking, they must be trained to perform conditional affective aspects of creative that is focused under a specified time period. The model of Wallas is used to measure and determine student’s position in creative thinking stages. Therefore the following study is conducted under the research aims of describing the profil of student’s creative thingking in solving mathematics problems based on Wallas model. RESEARCH METHOD This study uses a descriptive qualitative approach. Variable of this study is junior high school student’s creative thinking process in solving mathematics problem. Data were collected through tests of mathematics problem solving that are evaluated based the model of Wallas (2014). Subjects were students of class VII Sultan Agung Junior High School. They were selected through purposive sampling technique that is based on several criteria. The first criterion is based on the level of student’s creative thinking ability. The number of subjects at each level contains a minimum of two students. The second criterion is to choose students who have a good communication skill. The creative thinking process of students in solving mathematics problems is observed based on Wallas model. The procedure of this study as follows: 1) provide a test of mathematics problem solving to students; 2) analyze the results of students in solving mathematics problem to identify the abilities of students in creative thinking; 3) conduct interviews for students to know their creative thinking process in solving mathematics problem; 4) analyze the results of interviews. RESULTS AND DISCUSSION Students were given a mathematics problem in geometry which is open question where some alternative answers are possible. The results of the analysis of student’s answers in solving mathematics problems showed that 5 students have mathematics creative thinking ability in upper category (16,67%), 18 students have mathematics creative thinking ability in middle category (60%), and 7 students have mathematics creative thinking ability in lower category (23,33%). Furthermore, in each category of creative thinking ability, two representatives were interviewed. For students in upper category, they went through the stages of creative thinking process of Wallas very well. When students are trying to find alternative answers, they always taking into account the available time to convince whether the answer is correct. Controlling the answer is something that is very important to them. Figure 1 is an example of the creativity of the students to arrange several plane figures that have same areas with an area of a given plane figure. The first alternative, student made arrangement of four rectangles with a length is 15 and width is 4. In the second alternative, student can make arrangement some plane figures that consist of rectangle and triangle shown in Figure 1. Creative Thinking Process based on Wallas Model in Solving Mathematics Problem Maharani, Suketiyarno, & Waluya 180 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Figure 1. Sample of student work in upper category The results of interviews with student in upper category as follow. Question : With the available time, you made two alternative answers were correct. What did you do? Answer : First, I made a plane figure arrangement which has the same area with the given plane figure. After I found the right answer, I checked out a minute later and then I looked for another arrangement. Question : Did you found it difficult to determine the area of plane figure arrangement? Answer : No, I didn’t. Because I have ever did the same problem to find the area of some plane figures. For students in middle category, they can solve the problems by producing an alternative answer. It can be seen in Figure 2. Figure 2. Sample of the Student Work in Middle Category IJEME, Vol. 1, No. 2, September 2017, 177-184 IJEME ISSN: 2549-4996 181 The results of interviews with student in middle category as follow. Question : With the available time, you made two alternative answers were correct. What did you do? Answer : I lack of attention to the available time. I was busy tried to find the plane figure arrangement. Question : Did you have time to verify your answer? Answer : Because I was in a rush in doing, I didn’t have time to check again my answer. Finally, for students in lower category, they have difficulty to determine the plane figure arrangement. They just showed streaks calculation, but they haven’t made a specific plane figure arrangement in their answer sheet. The results of interviews with student in lower category as follow. Question : With the available time, you didn’t make an alternative answers were correct. What did you do? Answer : I feel confused. I had difficulty to find the plane figure arrangement. Result of the student work in upper category, on the answer sheet show that they can understand the given problem well. This means that the preparation stage has been done, where they have knowledge to solve the problem. In incubation stage, they tried to think of the arrangement in a few minutes. Until the illumination stage, they immediately make some alternative arrangements, apply their idea and made a plane figure arrangements. In verification stage, they look back at their works and then after they believe that the answer is right, they make the other plane figure arrangements. This is where students do creative thinking process stages Wallas well. As stated in the interview, that they have become familiar with the problem relating to the area of plane figure. This is inline with results of previous studies that students who arrived at the expected goals namely creative thinking ability, need to be given a habituation to each stage of creative thinking process. On the other hand, the events that occurred on the student in middle category, they did imperfection in the stages of creative thinking process based on Wallas. There is an emotional aspect that brings students to become unfocused in the creative thinking process. Students can understand the given problem in the preparation stage. But in incubation stage, students think a little longer to finally find an arrangement while they make calculation first in the other sheet. After finding the appropriate plane figure arrangement (reach the illumination stage), they make the arrangement in the answer sheet. When the answer has been completed, they didn’t check the results again and continue to think the other arrangement. This was because they require a longer time to find the arrangement. They have tried to make some arrangements, but the results are not right so they try to change the arrangement to another arrangement. The student’s work in lower ability of creative thinking indicates that students have difficulty in determining the area of plane figure arrangement. Student’s answer sheet only shows streak of calculation, but they haven’t made a specific plane figure arrangement. This is where students really have difficulties in solving mathematic problem. With more intensive scaffolding, teacher will be able to help them. Based on student’s mathematics creative thinking ability in solving mathematics problems, students suggest that their ability are still low. Majority of students did not Creative Thinking Process based on Wallas Model in Solving Mathematics Problem Maharani, Suketiyarno, & Waluya 182 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 fulfill all the stages of creative thinking process well enough. This was because they experienced obstacles to go beyond each stage in creative thinking process. Students in lower category have experienced obstacles at early stage, while students in middle category have experienced obstacles in the verification stage. Only students in upper category who were able to go beyond every stage of creative thinking well. The result of the deeper analysis of student’s work and interviews show that the process of student’s creative thinking in solving mathematics problem based on the model of Wallas are divided into three categories. It can be seen in Table 1. Analysis from the interviews showed that each student of three categories have different characteristics in each stage of Wallas model. This result shows that students in upper category do not have a difficulty in every stage of creative thinking process. They can solve the problem clearly and can provide various ways to answer. Students also can explain the solution clearly and detailly. Students in middle category in illumination stage show that they always try to solve a problem in any variety of ways. But sometime they make an error when applied their idea and they change or make a new solution, when in fact they can correct the solution without looking for the new solution. Table 1. Creative thinking process in solving mathematics problems based on wallas model Wallas’s Stage Preparation Incubation Illumination Verification Upper Category of Creative Thinking Ability Students understood the problems and could communicate information obtained well and use their own language Middle Category of Creative Thinking Ability Students were able to understand the problems, but in communicating information obtained students still use the problems language Lower Category of Creative Thinking Ability Students didn’t understand the problems and less able to communicate any information obtained Students tend to stop for a moment to observe a given problems and remember the material that they learn previously Students applied the idea to solve the problem with confidence and have the correct solution Students tend to stop for a moment to observe the given problems and try to make scribble on a blank paper Students applied the idea to solve the problem and the solution was generally true Students tend to stop a bit longer and think about what to do Students rechecked Students the solution. When rechecked finding an error they solution. IJEME, Vol. 1, No. 2, September 2017, 177-184 Students didn’t clear in implementing the idea to solve the problem and the solution obtained was wrong Student didn’t the recheck the When solution because IJEME ISSN: 2549-4996 try to correct the finding an error solution until get the they tend to right solution replace or make other solution 183 they feel they did the correct solution, so they didn’t correct the solution that still wrong Students in lower category from the beginning in preparation stage, they already have difficulties. They do not understand with the given mathematics problems and tasks. Beside, they do not have any information or knowledge that can be used to solve the problems. Consequently, students are unclear in implementing the idea to solve the problem and the solution obtained is also wrong. This result is in accordance with Potur & Barul (2009) that state students differ from one another in levels of motivation, backgrounds, attitudes towards teaching and learning, as well as the different responses to the classroom environment and certain instructional practices. So, students have different process in creative thinking too. In this study, students have differences in their creative thinking process. CONCLUSION Creative thinking ability is an ability that is important for everyone, not just when studying at school, but also when face with the world of work. In mathematics learning especially in geometry for junior high school, teachers and researchers can develop learning involves divergent thinking so as to enhance the student’s creative thinking ability. Teachers can provide more opportunities for students to explore answers or solution that can lead to a creative thinking process. Based on the model of Wallas (2014), has four stages of creative thinking process that include preparation, incubation, illumination, and verification. Results of this study, students were given a mathematics problem to solve in order to measure the ability of creative thinking. The results showed 1) 23,33% of students only reached preparation stage, called the low category, 2) 60% of students reached illumination stage though to arrive at this stage students takes a long time, called middle category, and 3) 16,67% of students have completed up to verification stage, called upper category. Students in lower category have difficulty to solve a mathematics problem. Therefore they needs more intensive guidances from teachers to assist them in performing each stage in the creative thinking process. For students in middle category teachers need to provide a little guidance and motivation to students when they did an error in solving the problem until they find the right solution. Students who have in upper category have been through the creative thinking process fluently, so teachers need to provide further enrichment materials. REFERENCES Aldous, C.R. (2007). Creativity, problem solving and innovative science: Insights from history, cognitive psychology and neuroscience. International Education Journal, 8(2), 176-186. Bloom. B.S., et al. (1956). Taxonomy of Educational Goals. Canada: David McKay Company, Inc. Bonaci, C.G., Mustata, R.V. & Lenciu, A. (2013). Revisiting bloom’s taxonomy of educational objectives. The Macrotheme Review, 2(2), 1-9. Creative Thinking Process based on Wallas Model in Solving Mathematics Problem Maharani, Suketiyarno, & Waluya 184 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Huitt, W. (2011). Bloom et al.’s Taxonomy of the Cognitive Domain. Educational Psychology Interactive. Valdosta, GA: Valdosta State University. IACBE. (2014). Bloom’s Taxonomy of Educational Objectives and Writing Intended Learning Outcomes Statements. USA: The International Assembly for Collegiate Business Education. Krathwohl, D.R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory Into Practice, 41(4), 212-218. Kwon, O.N., Park, J.S., & Park, J.H. (2006). Cultivating divergent thinking in mathematics through an open-ended approach. Asia Pacific Education Review, 7(1), 51-61. Maharani, H.R., & Sukestiyarno. (2015). Learning analysis based on humanism theory and mathematics creative thinking ability of students. International Journal of Education and Psychology, 1(2), 11-18. Pehkonen, E. (1997). The state-of-art in mathematical creativity. ZDM, 29(3), 63-67. Potur, A.A., & Barkul, O. (2009). Gender and creative thinking in education: A theoretical and experimental overview. Journal of ITU A| Z, 6(2), 44-57. Runisah, Herman, T., & Dahlan, J.A. (2017). Using the 5E learning cycle with metacognitive technique to enhance students’mathematical critical thinking skills. International Journal on Emerging Mathematics Education, 1(1), 87-98. Sari, E.P. & Sukestiyarno. (2014). Pemanfaatan media lingkungan dalam pembentukan karakter kemandirian siswa pada pembelajaran matematika model PBL pendekatan konstruktivisme. Proceedings of the National Conferenfe on Conservation for Better Life (NCCBL), Semarang State University, pp. 350-357. Wallas, G. (2014). The Art of Thought. England: Solis Press. Wessels, H. (2014). Levels of mathematical creativity in model-eliciting activities. Journal of Mathematical Modelling and Application, 1(9), 22-40. Widiastuti, M.M.H., & Sukestiyarno. (2014). Pemanfaatan lingkungan sekolah guna menunjang pembelajaran matematika dengan model cycle 7E dalam rangka membentuk karakter rasa ingin tahu siswa dan pemecahan masalah. Proceedings of the National Conferenfe on Conservation for Better Life (NCCBL), Semarang State University, pp. 331-338. Wulandari & Sukestiyarno. (2015). Development of ATONG based reference module for school geometry subject and analysis of mathematical creative thinking skills. Paper of the International Conference on Mathematics, Science, and Education (ICMSE), Semarang State University. IJEME, Vol. 1, No. 2, September 2017, 177-184 International Journal on Emerging Mathematics Education (IJEME) Vol. 1, No. 1, March 2017, pp. 1-10 P-ISSN: 2549-4996, E-ISSN: 2548-5806, DOI: http://dx.doi.org/10.12928/ijeme.v1i1.5507 Development of Student Worksheets to Improve the Ability of Mathematical Problem Posing 1,2Harry Dwi Putra, 2Tatang Herman, 1,2Utari Sumarmo 1STKIP Siliwangi, Jl. Terusan Jend. Sudirman No. 3 Baros, Cimahi Tengah 40526 2Universitas Pendidikan Indonesia, Jl. Dr. Setiabudhi No. 299, Bandung 40154 Email: [email protected] Abstrak Aktivitas mengajukan masalah (problem posing) dan menyelesaikannya merupakan bagian penting dari matematika, sehingga problem posing merupakan aktivitas yang sangat berperan dalam berpikir matematis dan menjadi bagian yang penting dalam pemecahan masalah. Jenis penelitian ini adalah research and development yang bertujuan untuk mengembangkan lembar kerja siswa mengenai aturan pencacahan dengan pendekatan scientific disertai strategi what if not. Metode penelitian terdiri dari: studi pustaka, observasi, wawancara; pengembangan bahan ajar, penilaian dari dua ahli, dan uji coba terbatas pada siswa kelas XI di SMAN 2 Cimahi, SMAN 3 Cimahi, dan SMAN 4 Cimahi. Berdasarkan hasil penelitian, diperoleh lembar kerja siswa yang memuat aktivitas mengamati, menanya, mencoba, menalar, dan menyimpulkan. Pada aktivitas menanya menggunakan strategi what if not dengan cara merubah data, menambah data, mengubah data dengan pertanyaan yang sama, atau mengubah pertanyaan dengan data yang sama. Dalam lembar kerja disajikan masalah kontekstual yang sesuai dengan pengalaman siswa. Setelah dilakukan uji coba menggunakan lembar kerja, peningkatan kemampuan mathematical problem posing siswa pada ketiga sekolah berada pada kriteria sedang. Kata Kunci: lembar kerja siswa, pendekatan scientific, strategi what if not, mathematical problem posing Abstract Activities posing and solving a problem are important parts in mathematics, so the problem posing is an activity that was instrumental in mathematical thinking and become an important part in solving the problem. This type of research is research and development that aims to develop student worksheets through scientific approach with what if not strategy. The research method consists of stages: literature study, observation, interviews, development of student worksheets, assessment of two experts, and restricted trial to the eleventh grade students in SMAN 2 Cimahi, SMAN 3 Cimahi, and SMAN 4 Cimahi. Based on the findings, it was concluded that the student worksheets through scientific approach with what if not strategy load some activities including observing, questioning, trying, reasoning, and concluding. The activity of questioning using what if not strategy involves changing data, adding data, changing data with the same question, or changing question with the same data. The student worksheets also presented some contextual problems in accordance with the experience of students. After tested using worksheet, the improvement of student mathematical problem posing in the three schools was in moderate criteria. Keywords: student worksheets, scientific approach, what if not strategy, mathematical problem posing How to Cite: Putra, H.D., Herman, T., & Sumarmo, U. (2017). Development of student worksheets to improve the ability of mathematical problem posing. International Journal on Emerging Mathematics Education, 1(1), 1-10. http://dx.doi.org/10.12928/ijeme.v1i1.5507. INTRODUCTION The heart of mathematics is to pose problems and solve them (Brown & Walter, 1990). Therefore, the problem posing activity become an important role in Received December 28, 2016; Revised February 23, 2017; Accepted February 27, 2017 2 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 mathematical thinking and become an important part on problem solving. Based on some findings, Winograd (1997) suggested that giving assignments to the students to create questions could improve the ability to solve problems and their attitudes toward mathematics. In line with this finding, English (1998) stated that the problem posing could improve thinking skills, problem solving, attitude, confidence in solving problems, and generally contribute to the understanding of mathematical concepts. To acquire the ability of problem solving, students should have many experiences in solving various problems (Herman, 2007). Mathematics learning in senior high school should load tasks to formulate mathematical problems based on various situations, both inside and outside mathematics, organize and find conjecture, and learn to generalize and extend the problem through problem posing (NCTM, 2000). To develop the ability of mathematical problem posing, we required the ability to creating new questions, creating new opportunities, and looking at old questions from a new angle (Ellerton & Clarkson, 1996). This opinion is supported by Silver (2013) who states that the relationship between creativity and posing problems does not stand alone but affects each other. After made questions, the students were asked to check the accuracy of the questions answered. These activities could train the students to think reflectively. Problem posing is not only seen as a purpose of learning but also as a tool that can provide experiences for students to find and to create mathematical problems. This experience is still a bit possessed by students. According to the findings of Van Harpen & Sriraman (2013) advanced high school students had trouble posing good quality and novel mathematical problem. Joo & Han (2016) revealed that some senior high school students had difficulties in posing problem or limited understanding of that. Igor (2016) found that the problem posing performance of senior high students and the teachers was the low quality of the created problems. Therefore, the ability of problem posing should be trained to the students in order to develop the ability to think, to be skillful to troubleshoot, and expand the understanding of the concept. If the teacher presents a challenging problem and requires students to think, then the situation can provide greater opportunities for students to develop critical thinking. In the curriculum of 2013 emphasizes on a contextual approach becomes the process approach as strengthening of the use of scientific approach. The scientific approach is one of the approach that facilitates students to acquire knowledge and skills based on a scientific method: observing, questioning, reasoning, trying, and forming networks (concluding, presenting, and communicating). Students are directed to process knowledge, discover and develop their own concepts with regard to the subject matter so that it provides an opportunity for students to cultivate high-level thinking skills (Kemdikbud, 2013). Brown & Walter (1990) stated that generally mathematical thinking begins with given statements so that we just trained to resolve the issue of the statement. However, it should be better if we give another statement rather than just accept the statement. In mathematical thinking, posing a problem is better than just finishing that problem. One of the learning strategies that can develop the high-level ability of mathematical thinking is what if not strategy. This strategy can develop mathematical problem posing abilities through activities of students in analyzing problems, contrasting conditions on the problem, and checking the correctness of completion. Based on the explanation above, it seems that the ability of mathematical problem posing for students is so important that teachers need to develop student worksheets through appropriate learning approaches and strategies in order to IJEME, Vol. 1, No. 1, March 2017, 1-10 IJEME ISSN: 2549-4996 3 improve the ability of the high-level mathematical thinking. The research questions of this study are formulated as follows: How to develop student worksheets through a scientific approach with what if not strategy to improve the students’ ability of mathematical problem posing? What is the impact of the use of student worksheets through a scientific approach with what if not strategy on the students’ ability of mathematical problem posing? RESEARCH METHOD This research is a research and development that produces student worksheets through scientific approach with what if not strategy to improve the ability of mathematical problem posing for students in senior high school. The procedures of this research include (Sukmadinata, 2012): a field survey to obtain information from the teachers in a certain school about mathematical problem posing abilities of the students. Literature review to examine the student worksheets that will be developed through a scientific approach and strategy of what if not. Drafting of student worksheets that will be assessed by a team of experts. The worksheets that had been validated were then tested to eleventh grade students from three schools, which have high, medium, and low criteria of senior high school namely SMAN 2 Cimahi, SMAN 3 Cimahi, and SMAN 4 Cimahi. Research procedure shown in the following Figure 1. Figure 1. The Procedure of Learning Materials Development Research Development of student worksheets to improve the ability of mathematical problem posing Putra, Herman, & Sumarmo 4 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 RESULTS AND DISCUSSION Based on literature review, it can be obtained some indicators of the ability of mathematical problem posing to the material of enumeration rules. In making question dealing with the ability of mathematical problem posing for enumeration rules pertaining to matter, it refers to the following indicators (Sumarmo, 2015). Draw up a new question regarding to the problem of enumeration rules. Express a problem into another form that has the same meaning with the material of cyclic permutation. Ask questions of a series of mathematical semi-structured information with respect to the ability to think reflectively on the material of permutations. Specify the questions about the material of combination into question parts. Ask questions before, during, and after solving problems with regard to the material of chances. According to interviews with teachers in SMAN 2 Cimahi, SMAN 3 Cimahi, and SMAN 4 Cimahi, students still had difficulty in solving the problems of permutations and combinations within the rules enumeration. They were confused on using permutation or combination formula to solve the problems. Presentation of the material from the book makes students follow the example of completion of the given problem, so that when the questions were changed, they become confused to define the concepts to use whether it is permutation or combination. In lessons, students rarely ask questions. That was only teachers who posed the question to the students. This shows that the students' ability to pose a problem was undeveloped. One of the appropriate approaches is scientific approach with what if not strategy. In the opinion Brown & Walter (1990) to develop the ability of problems posing, we can use a what if not strategy by changing the data in question, adding data on the problem, changing the data with the same question, or changing the question with the same data on the problem. Based on observations and interviews conducted to teachers at those three schools, it can be concluded that the worksheets need to be developed through a scientific approach with what if not strategy to improve the ability of mathematical problem posing of high school students. The initial design of student worksheets includes three parts. The front part consists of a title page, preface, and a table of contents. Displays the title pages of the student worksheets are made. The body section consists of a crosshead of the enumeration rules that represent the content of the chapter: rule multiplication, permutations, combination, Binomial Newton, and Chance. At the beginning of the presentation of the material given the problems that exist in daily life of the students were resolved by the given instructions to find the concept. The conclusion sections, students give reflection and evaluation of the presentation material on worksheets that they used to work on in order to make improvements. Student worksheets were adjusted to the scientific approach that includes activities to observe, ask, try, reason, and concludes. These activities can vary the sequence. Figure 2. Observation Activity on Worksheets IJEME, Vol. 1, No. 1, March 2017, 1-10 IJEME ISSN: 2549-4996 5 In the observation, problems were presented as an introduction to the concepts to be learned. In the figure 2, students observe the problems regarding the bank officer who want to print the costumer’s queue number consisting of three numbers of 1, 2, 3, and 4. The question is how much the queue number choices can be made from the first three digits? How much the queue number choices can be made from the four numbers are available? Figure 3. Try Activity on Worksheets In the stage of a trying, students were asked to solve problems in worksheets. Students are given instructions in solving problems. A lot of the order queue number consisting of three numbers of 1, 2, and 3 are 6 composition. Whereas, a lot of the order number of the queue consisting of three numbers of 1, 2, 3, and 4 are 24 order. Based on the instructions, students try to adjust the answer. When students answered not in accordance with the instructions, they will reflect on their mistakes. Figure 4. Reasoning Activity on a Worksheets In the reasoning activity, students are directed to find an answer through the concept of factorial. Students examine the truth of the answers by providing an explanation. The number 6 can be converted into a pattern of 3 2 1. Any number Development of student worksheets to improve the ability of mathematical problem posing Putra, Herman, & Sumarmo 6 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 3 2 1 . Based on the concept 1 3 2 1 3! of factorial, 3 2 1 pattern can be written as 3! so . Karena 1! equal to 1 1! 3! 3! . If students can explain the truth of the answers, it means that (3 3)! so 1! (3 3)! the students already understand the concept of factorial. divided by 1 the result is the number itself, so 3 2 1 Figure 5. Questioning Activity on a Worksheets In the questioning activity, students ask two more questions about these problems. Then, students choose one question and answer it. Students can use what if not strategy to ask questions. When a bank officer would like to print the costumer’s queue number consists of 4 numbers from 1, 2, 3, 4, and 5. The questions are: 1. How much the queue number choices can be made from the first four digits? 2. Is it true that a lot of the order that can be made from 5 digits available is 120 compositions? Figure 6. Concluding Activity on a Worksheets IJEME, Vol. 1, No. 1, March 2017, 1-10 IJEME ISSN: 2549-4996 7 In the conclusion activity, students make inferences about the concept was studied. Students understand that many compositions can be formed from 3 different numbers of 3 numbers available are called permutations 3 elements are indicated by 3! the symbol P(3,3) 3 P3 P33 3! . Many compositions can be formed from 3 (3 3)! different numbers of 4 number available are indicated by the symbol 4! P(4,3) 4 P3 P34 . In conclusions, a lot of permutations of k elements is k ! (4 3)! and a lot of permutations of k elements take from n elements available is n! . P (n, k ) n Pk Pkn (n k )! Assessment of student worksheets according to a team of experts from the aspect of the contents had fulfilled the five elements in the scientific approach: observing, questioning, trying, reasoning, and concluding in teaching the concept of problem. From the display of student worksheets, it looks simple. Display of student worksheets had already developed a mathematical problem posing abilities of students in asking questions about new problem or developing a given problem. Based on the results of the assessment team of experts, these materials were already eligible tested on students. Here is presented the assessment of the experts. Tabel 1. Feasibility test result of the student worksheets Feasibility Criteria (%) No. Experts Content Display Language 1. Validator 1 83,33 85,94 81,67 2. Validator 2 79,19 82,81 80,00 Mean 81,25 84,38 80,83 Based on Table 1, it seems that the average ratings of student worksheets by both the validator based on the criteria of content and language is between 63.50%83.00% that means it is feasible. While the criteria of display is above 83,00% which means it is more feasible. Therefore, worksheets are feasible to be tested on students. Student worksheets that had been assessed shown to teachers at three schools to get suggestion for further development. The worksheets can be tested, which is in eleventh grade students in Mathematics and Science classes of SMAN 2 Cimahi, SMAN 3 Cimahi, SMAN 4 Cimahi. Student worksheets that had been validated are given to students in learning activity nine meetings. Students were given an explanation regarding to the presentation of worksheets so that they can solve the problems properly. There are five scientific activities that must be done by students: look into the subject, ask questions about the problems with the strategy of what if not, try to solve the problem, reason against the troubleshooting steps, and make conclusions from the material they had learned. After finishing studying all the material of enumeration rules in student worksheets, they were asked to fill a scale of opinion regarding to the presentation of the worksheets they had learned. Percentage score of opinion scale of students from three schools toward the student worksheets were 67.86%, 65.79% and 65.14%. These results were between 61.00% - 80.00% which means that students had positive opinion toward worksheets. Students agree that the worksheets contain the task of asking questions. Student worksheets can increase their confidence to draw the questions and solve them. Table Development of student worksheets to improve the ability of mathematical problem posing Putra, Herman, & Sumarmo 8 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 2 below presented the results of the pretest, posttest, and enhancement (N-gain) of the students. Ideal Maximum score is 14. Tabel 2. Result of pretest dan posttest of mathematical problem posing ability School Total Pretest Posttest N-gain Criteria SMAN 2 Cimahi 31 4,31 10,73 0,67 Moderate SMAN 3 Cimahi 35 4,51 10,48 0,63 Moderate SMAN 4 Cimahi 37 4,29 10,16 0,60 Moderate Based on Table 2, it is shown that an improvement of the ability of mathematical problem posing of the students from the third grade is moderate. Students were not accustomed to drafting a new question, stating the problem in another form with the same meaning, asking questions with regard to determining and verifying the relevant data, scanting details into the form of parts, as well as asking questions before, during, and after troubleshooting. Through this student worksheets, it can be said that the ability of mathematical problem posing of students was increased quite well. CONCLUSION Based on the results of the assessment team of experts and the restricted trial of student worksheets, it can be concluded that student worksheets contained the activities of observing, questioning, trying, reasoning, and concluding in accordance with the principles of the scientific approach. What if not strategy was applied to the activities of questioning, by changing the data, adding data, or changing the question. The designs of student worksheets consist of three parts. Beginning consists of a title page, preface and table of contents. The content section containing the title of the chapter on the rules of the enumeration and contextual problems that exist in the students’ life. Closing as a reflection and evaluation of the material presented on student worksheets for the next revision. Student worksheets could increase the ability of mathematical problem posing in the moderate criteria. ACKNOWLEDGEMENT The author would like to extend thanks to the chair and staffs of the Ministry of Research Technology and Higher Education (KEMENRISTEK DIKTI) who has helped funding this research. REFERENCES Brown, S.I., & Walter, I. (1990). The Art of Problem Posing (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Ellerton, N.F., & Clarkson, P.C. (1996). Language Factor in Mathematics Education. Alphen aanden Rijn: Kluwer Academic Publisher. English, L.D. (1998). Children's problem posing within formal and informal contexts. Journal for Research in Mathematics Education, 29(1), 83-106. Herman, T. (2007). Pembelajaran berbasis masalah untuk meningkatkan kemampuan berpikir matematika tingkat tinggi siswa sekolah menengah pertama. Educationist, 1(1), 47-56. Igor, K. (2016). Considerations of aptness in mathematcial problem posing: students, teacher, and expert working on billiard task. Far East Journal of Mathematical Education, 16(3), 243-260. IJEME, Vol. 1, No. 1, March 2017, 1-10 IJEME ISSN: 2549-4996 9 Joo, H., & Han, H. (2016). The analysis of middle school students' problem posing type and strategies. The Mathematical Education, 55(1), 73-89. Kemdikbud. (2013). Pendekatan Scientific (Ilmiah) dalam Pembelajaran. Jakarta: Pusbagprodik. Kontorovich, I. (2016). Considerations of aptness in mathematcial problem posing: students, teacher, and expert working on billiard task. Far East Journal of Mathematical Education, 16(3), 243-260. NCTM. (2000). Principles and Standards for School Mathematics. Reston, VA: National Council of Teacher Mathematics. Silver, E.A. (2013). Problem posing reseach in mathematics education: looking back, looking around, and looking ahead. Educational Studies in Mathematics, 83(1), 157-162. Sukmadinata, N.S. (2012). Metode Penelitian Pendidikan. Bandung: PT Remaja Rosdakarya. Sumarmo, U. (2015). Mathematical Problem Posing: Rasional, Pengertian, Pembelajaran, dan Pengukurannya. Retrieved from STKIP Siliwangi: http://utarisumarmo.dosen.stkipsiliwangi.ac.id/files/2015/09/Problem-Posing-MatematikPengertian-dan-Rasional-2015.pdf Van Harpen, X.Y., & Sriraman, B. (2013). Creativity and mathematical problem posing: an analysis of high school students' mathematical problem posing in China and the USA. Educational Studies in Mathematics, 82(2), 201-221. Winograd, K. (1997). Ways of sharing student-authored story problems. Teaching Children Mathematics, 4(1), 40-49. Development of student worksheets to improve the ability of mathematical problem posing Putra, Herman, & Sumarmo 10 IJEME, Vol. 1, No. 1, March 2017, 1-10 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 ISSN 2087-8885 E-ISSN 2407-0610 Journal on Mathematics Education Volume 11, No. 1, January 2020, pp. 59-76 ELEMENTARY PRESERVICE TEACHERS’ KNOWLEDGE, PERCEPTIONS AND ATTITUDES TOWARDS FRACTIONS: A MIXED-ANALYSIS Roslinda Rosli1, Dianne Goldsby2, Anthony J. Onwuegbuzie3, Mary Margaret Capraro2, Robert M. Capraro2, Elsa Gonzalez Y. Gonzalez4 1Universiti Kebangsaan Malaysia, UKM Bangi, Malaysia A&M University, College Station, Texas, United States 3Sam Houston State University, Huntsville, Texas, United States 4University of Houston, Texas, United States Email: [email protected] 2Texas Abstract Previous research has shown knowledge, perceptions, and attitudes are essential factors during mathematics classroom instruction. The current study examined the effects of a 3-week fraction instructional unit using concrete models, problem-solving, and problem-posing to improve elementary preservice teachers’ knowledge, perceptions and attitudes towards fractions. A quasi-experiment design was implemented to gather data via closed-ended, open-ended, and essay tasks from a convenience sampling of 71 female elementary preservice teachers during pre- and post-assessments. The study discovered that the select preservice teachers were weak in the content knowledge specifically on unit-whole, part-whole, equivalent area, arithmetic operations, and ordering fractional values. In contrast, the incorporation of concrete models, problem-solving and problemposing was effective in improving the preservice teachers’ level of pedagogical content knowledge, perceptions and attitudes towards fractions. Implications of the results and suggestions are discussed. Keywords: Elementary School, Problem Posing, Teacher Preparation Program, Preservice Teachers, Mixed Methods Abstrak Penelitian sebelumnya menunjukkan bahwa pengetahuan, persepsi, dan sikap merupakan faktor penting dalam pembelajaran matematika. Penelitian saat ini meneliti tentang pengaruh dari pembelajaran pecahan selama kurun waktu 3 minggu menggunakan model konkret, pemecahan masalah, dan problem-posing untuk meningkatkan pengetahuan, persepsi, dan sikap calon guru sekolah dasar terhadap materi pecahan. Sebuah desain eksperimen semu diimplementasikan untuk mengumpulkan data melalui permasalahan dalam bentuk soal closed-ended, open-ended, dan uraian dari pengambilan sampel yang representative dari 71 calon guru wanita sekolah dasar selama pra dan pasca penilaian. Studi ini menemukan bahwa calon guru terpilih lemah pada konten materi pecahan, khususnya pada unit-whole, part-whole, area yang sama, operasi aritmatika, dan nilai pecahan berurutan. Sebaliknya, penggabungan model konkret, pemecahan masalah, dan problem-posing adalah efektif dalam meningkatkan level pengetahuan konten pedagogis calon guru, persepsi, dan sikap terhadap materi pecahan. Implikasi hasil dan saran lebih lanjut dibahas dalam tulisan ini. Kata kunci: Sekolah Dasar, Problem Posing, Program Persiapan Guru, Calon Guru, Metode Campuran How to Cite: Rosli, R., Goldsby, D., Onwuegbuzie, A.J., Capraro, M.M., Capraro, R.M., & Gonzalez, E.G.Y. (2020). Elementary preservice teachers’ knowledge, perceptions, and attitudes towards fractions: A mixedanalysis. Journal on Mathematics Education, 11(1), 59-76. http://doi.org/10.22342/jme.11.1.9482.59-76. Knowing numbers and operations is a cornerstone of mathematics education in the school curriculum (National Council of Teacher of Mathematics [NCTM], 2000). Young students should have acquired a conceptual understanding of number systems, their structures, and properties during classroom instruction (Lamon, 1999). Fractions are considered central concepts for school mathematics but have conventionally been challenging and cumbersome for teachers to deliver as well as for students to learn (Barnett-Clarke, Fisher, Marks, & Ross, 2010; Lin, 2010). An incomplete understanding of fractions 59 60 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 can eventually affect individuals’ learning in subsequent mathematical areas such as algebra, trigonometry, and calculus and related disciplines (Barnett-Clarke et al., 2010). Numerous previous studies have emphasized the impact of diverse classroom instruction (e.g., traditional, concrete models, web-based, one-on-one) on student learning of fractions (Lin, 2010; Newton, 2008; Osana & Royea, 2011). However, there has been scant research measuring the potential benefit of an instructional practice that integrates concrete models for developing preservice teachers’ conceptual understanding of fractions (Sarama & Clements, 2009). In this study, we focused on measuring the integration of Mathematics TEKS Connection module with concrete model instructional practices in facilitating preservice teacher knowledge construction, perceptions and attitudes towards fractions during the teacher preparation programs. A body of literature has documented that a majority of in-service teachers and preservice teachers have limited profound knowledge for teaching mathematics (Ball, 1993; Hill, 2010; Ma, 1999; Newton, 2008; Timmerman, 2004). In addition, White, Way, Perry and Southwell (2005) and Llinares (2002) stated preservice teachers' belief or perceptions is reflected in their action or attitudes during teaching and learning that influence the classroom instructional practices. Despite this evidence, little is known about the nature of knowledge for teaching fractions, perceptions and attitudes that teachers should have nurtured during teacher preparation, the place where teachers should have acquired their teaching repertoire. Teacher Knowledge Ball (1993) and Ma (1999) stated that teachers’ content knowledge could be a possible aspect affecting classroom instruction. Meanwhile, Shulman (1986) noted content knowledge alone is not sufficient; pedagogical content knowledge is also significant for making the learning of mathematical concepts understandable. Shulman (1986) pointed out that content knowledge is solely an amount of mathematics knowledge that one should have for teaching a particular concept. It includes the conceptual and procedural understanding of specific mathematical ideas (Shulman, 1986). Many researchers argue the importance of teacher knowledge in making teaching and learning of mathematical content meaningful. When teachers do not possess in-depth knowledge of a particular concept (Ball, Thames, & Phelps, 2008) and do not know how to represent the idea and to make it comprehensible and understandable (Shulman, 1986), they often fail to deliver the concept for students’ understanding (Barnett-Clark et al., 2010). In contrast, pedagogical content knowledge includes the way a teacher represents fractions to facilitate student learning by using appropriate models, analogies, illustrations, examples, explanations, and demonstrations (Shulman, 1986). Also, teachers must be aware of students’ knowledge of fractions and know the strategies for reorganizing students’ understanding appropriately (Shulman, 1986). Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 61 Perceptions and Attitudes Previous research showed preservice teachers entered teacher preparation programs with preexisting views grounded mainly on their background and positive or negative experiences in learning mathematics (Timmerman, 2004; White et al., 2005). These preexisting perceptions are resistant to change and difficult to break (Kane, Sandretto, & Heath, 2002). White et al. (2005) claimed an intervention is needed to stop a cycle of negativity perceptions and attitudes towards mathematics especially for preservice teachers who in turn influence the formation of student attitudes. According to White et al., negative perceptions could create negative teaching ways, that can successively affect students’ beliefs, attitudes and learning outcomes. Thus, teacher educators should perceive preservice teacher perceptions and attitudes expressly for utilizing the potential ways for knowledge construction and stimulating learning. Multiple Tasks and Concrete Models Appropriate instructional materials and mathematical tasks in classrooms are indeed crucial for helping students grasp abstract concepts for knowledge construction (Cramer & Wyberg, 2009; Sarama & Clements, 2009). The incorporation of problem-solving and problem-posing tasks can help teachers to enhance students’ understanding of fraction with the assistance of concrete models such as fraction strips, and paper-folding. For instance, when students are asked to generate a problem for this division of fractions, 3 4 1 8 ÷ = 𝑛 (Sowder, Philipp, Armstrong, & Schappelle, 1998), they can use a concrete model to discover fraction size (Empson, 2002) and the relationships between the 1 4 1 and the 8. The teacher can examine students’ understanding by asking students to explain the meaning of 3 4 1 ÷ 8 as illustrated in Figure 1. Based on the representational model of Figure 1, students should be able to make connections to their prior knowledge about fractional numbers and find the answer to the problem. Figure 1. Models for Division of Fraction 62 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 Teachers may assist students’ learning during the process of operating the models. A student 1 3 must be able to grasp the concepts and ponder the underlying association on six sets of 8 in 4. Based on the necessary knowledge, the student could think of a fraction situation that fits the given equation and then teachers would ask the students to share their situations with others in the class. METHOD In utilizing both quantitative and qualitative data and rigorous methodological research techniques, we adapted close-ended and open-ended problem-solving and problem-posing tasks to shed light on the research question: What was the effect of the instructional unit of developing conceptual understanding of fractions with concrete models on the levels of elementary preservice teachers’ content knowledge, pedagogical content knowledge, perceptions and attitudes towards learning and teaching fractions? The pragmatist approach (Onwuegbuzie, Johnson, & Collins, 2009) drove the study that involved administering two pre- and post-assessments containing close-ended items and open-ended questions. Pragmatist researchers believe that there are many useful ways of seeking knowledge, including mixing quantitative and qualitative data and methods (Onwuegbuzie et al., 2009). A quasi-experimental research method was utilized for gathering both quantitative and qualitative data that focused on a onegroup pretest-posttest design (Shadish, Cook, & Campbell, 2002). The qualitative component of the study was based on the constructivist-naturalistic paradigm (Lincoln & Guba, 1985). We attempted to assess the preservice teachers’ understanding and attitudes about fractions by examining the open-ended responses and seek “individual and collective reconstructions that may unite around consensus” (Onwuegbuzie et al., 2009, p. 122). It was hoped that we could assess the treatment integrity (e.g., fidelity score) of the fraction instruction using both quantitative and qualitative instruments (Collins, Onwuegbuzie, & Sutton, 2006) to reduce any implementation biases (Onwuegbuzie, 2003). Besides, mixing quantitative and qualitative data can provide significant enhancement of the findings for generalizability purposes (Collins et al., 2006). The study employed the use of a convenience sampling technique to select the participants who were willing to be part of the study (Collins, Onwuegbuzie & Jiao, 2007). The instructor, Dr X who is an experienced professor, has been teaching elementary methods courses for more than ten years at one of the public universities in Texas. The method course was compulsory for the completion of the undergraduate degree with certification in the EC-Grade 6 Core Subjects/Generalist program. The participants were 71 female preservice teachers who were pursuing a Texas teacher certification in elementary school classrooms. They were from two class sections of the experienced professor in the Fall semester. The course comprised of one hour and 15 minutes of 16-week face-to-face meetings together with an online module through the Blackboard Learning System (eLearning) of the university. Concerning ethical considerations, permission was granted to perform the study over the Institutional Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 63 Review Board (IRB) of the university and an information sheet was provided to the participants informing them that the study would run as regular class session. The times for both class sections were back to back and the professor taught the same content and used similar instructional strategies. Close-ended items from the Learning Mathematics for Teaching [LMT] (2008) were utilized for examining content knowledge and pedagogical content knowledge. Only 11 close-ended were selected based on content-related validity to fractional concepts (Measure A) under investigation. In addition, we adopted five close-ended items on geometry (Measure B) to examine the non-equivalent dependent variable for improving the design. This instrument was made available to participants using an online survey tool, Qualtrics, and was piloted to preservice teachers in the other class sections of the methods course. The internal consistency scores (Cronbach’s alpha) were .74 for fractions and .73 for geometry using the Statistical Program for Social Science (SPSS) software version 21.0 (IBM Corp., 2013). Additionally, nine open-ended items were selected from Sowder et al., (1998), five questions were used to measure content knowledge and four were utilized to assess pedagogical content for teaching fractions. Two professors reviewed these open-ended questions to determine the contentrelated validity such as the relevancy and appropriateness of the tasks (Collins et al., 2006). The openended instrument was piloted and no changes were made to the instrument and the administration time of the present study. The participants received three weeks of classroom instruction (each week for 4 hours) on basics fraction concepts including addition, subtraction, multiplication, and division. The classroom instruction was comprehensively concentrated on concrete models that helped preservice teachers discover and construct basics mathematics knowledge based on the constructivist theory. Four to five preservice teachers were grouped into a small station (table) and concrete models such as tangrams, fraction bars, fraction circles, fraction towers, fraction strips, and pattern blocks were made available in every class session. During class instruction, Dr. X adopted a meaningful teaching approach through demonstrations and problem-posing activities that involved preservice teachers’ use of modeling to develop conceptual knowledge of fractions. Preservice teachers were given various fraction problems to create a scenario for and model with manipulatives or drawings with their partners. Problems were such as as 1 2 1 + 4, 2 3 1 −4, 1 2 3 3 1 1 × 4 , 4 × 2 , 4 ÷ 2 , and 1 2 ÷ 4. In addition to classroom instruction, preservice teachers engaged and participated actively in online modules, which utilized multiple instructional strategies including web-based activities, videos, and readings. The modules were developed and field-tested that followed the Mathematics Texas Essential Knowledge and Skills (TEKS) Connection [MTC Project]. It was a part of the Texas Math Initiative and represented a partnership between the Texas Education Agency (TEA) and the university. With three different bands (K-4, 5-8, 9-12), it covered some specific and vital mathematics concepts (e.g., place value, fractions) for each grade level. The modules were made available for future teachers use from http://mtc.tamu.edu/home.htm?intro-pre.htm. 64 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 The fraction assessments were administered sequentially online during class periods in October 2011. A week before the professor began the instruction on fractions, the preservice teachers were asked to access the Qualtrics survey tool and respond to 16 multiple-choice items by choosing their best options. Then, during one class period, a graduate assistant administered the open-ended tasks wherein participants were required to write their responses on an answer sheet. The fraction instruction took place during three weeks wherein unstructured observations were made and field notes were taken to capture all potentially relevant phenomena. The graduate assistant was an observer-as-participant (Johnson & Christensen, 2012) wherein the primary role was to collect related information (e.g., setting, interactions, and subtle factors) concerning fraction instruction (Erlandson, Harris, Skipper, & Allen, 1993). A week after Dr. X completed the fraction instruction, the preservice teachers were required to login to the Qualtrics instrument and answer the same items as on the pre-assessment. Similarly, the graduate assistant administered the same open-ended assessment to the participants. The close-ended items were scored based on their correctness, one point for a exact answer or zero point for an inappropriate one. Data were stored in the SPSS software (SPSS Inc., 2007) and points for items were totaled up for content knowledge (i.e., Item 1, 3, 4, 6, 7, 10, and 11) and pedagogical content knowledge (Item 2, 5, 8, and 9). Next, the open-ended items were transformed quantitatively where each pre- and the post-written response was assessed and coded into a numerical value (Teddlie & Tashakkori, 2006) based on the degree of accomplishment, zero (lowest) - four points (highest). For inter-rater agreement percentage, two independent raters graded seven identical scripts and attained an 83.3% agreement. The raters discussed to resolve any disagreement. Based on the data collection, a few statistical analyses were run and effect sizes for pre-post contrast were calculated. Then, we analyzed the written responses and observation data through a constant comparison analysis (Glaser, 1965) for a more in-depth and better understanding of the elementary preservice teachers’ strategies (Lincoln & Guba, 1985) for solving fractional tasks. The essay parts were unitized using the QDA Miner 3.2 (Provalis Research, 2009) following the classical content analysis method (Onwuegbuzie & Teddlie, 2003). The written essays were examined, underlined into chunks, and unitized into smaller significant parts (codes). The codes that were frequently appear were counted to symbolize essential concepts of preservice teachers’ perceptions and attitudes towards fractions. RESULTS AND DISCUSSION Based on the quantitative results and qualitative findings, we hoped to make external (statistical) generalizations regarding the level of fractional understanding to the entire population from where the participants were conveniently drawn. In the present study, the differences from pretest to posttest scores on the problem-solving and problem-posing tasks were utilized to observe the effect of the fraction instruction that focused on concrete models. These employed mixed data analyses to measure the change in the levels of elementary preservice teachers’ content knowledge, pedagogical content Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 65 knowledge, perceptions and attitudes regarding learning and teaching fractions. Analysis of Quantitative Data This quasi-experimental design utilized a one-group pretest-posttest design to assess the causeand-effect relationship of fraction instruction (treatment) and preservice teachers’ knowledge and belief about fractions. For reducing threats to validity of the experimental design, we adapted six geometry items as the nonequivalent dependent variable (measure B) in addition to 11 fraction items for measure A. The results showed the preservice teachers’ scores of fraction knowledge (measure A) changed statistically significantly, t(59) = 3.50, p = .001 with a mean gain effect size of .32 from pretest to posttest possibly because of the treatment of fraction instruction. No statistically significant change for geometry knowledge (measure B) was noted from pretest (M = 5.13) to posttest (M = 5.05). According to Shadish et al. (2002), this indication is useful for ruling out the possibility of a cause-and-effect relationship in a research study. The results of the close-ended assessment included 60 completed responses from the select preservice teachers. Eleven answer scripts were eliminated due to an invalid/incomplete pretest or posttest. The t-test value indicates that the mean score for fraction knowledge on the posttest was significantly greater than the mean score on the pretest. Specifically, the results from the close-ended items revealed statistically significant differences in means scores for both preservice teachers’ content knowledge [t(59) = 2.14, p = .037] and pedagogical content knowledge [t(59) = 2.87, p = .006]. The standardized mean gain effect sizes were 0.19 and 0.38 respectively. The results indicate that the preservice teachers’ content knowledge and pedagogical content knowledge improved during instruction as measured by using close-ended items. A review of the item analysis showed that preservice teachers scored a lower percentage for content knowledge specifically on Items 4, 6, 7 10, and 11 with less than 40% answering correctly on both pretest and posttest. Also, we found that preservice teachers had the most difficulty on Item 4 with only 10% correct on the pretest and posttest. They were not able to interpret and to analyze different situations given in the question that related to the unit-whole of a fraction. The results supported Lamon’s (1999) argument that most of the classroom instruction failed to present the unit-whole fraction that contributes to students’ misunderstanding of the concept. However, the majority of preservice teachers were able to answer correctly the part of Items 2, 5, and 8 that involved their knowledge of problem posing. Besides, the most significant differences between the percentage of scores on the pretest and posttest were found on Items 6, 2 (item b only), 8 (all sub-items), and 9. Analysis of Qualitative Data As mentioned in the previous section, the qualitative data were mainly collected from preservice teachers’ responses to the open-ended assessment consisting of five fraction content knowledge items and four fraction pedagogical content knowledge items. Based on 71 completed responses, each item 66 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 was assessed and transformed into a numerical value (0-4 points) that can be analyzed statistically; then the subtotals were calculated for content knowledge and pedagogical content knowledge. Overall, the Wilcoxon signed-rank test revealed that a 3-week instruction did not elicit a significant improvement statistically in the level of elementary preservice teachers’ content knowledge [Z = -2.25, p = .024, r =.21]. Indeed, the median content knowledge score was 2.0 both pre- and post-assessment. The statistical analyses of each item under content knowledge were utilized and the Bonferroni adjustment with α = .05/5 = .01 was applied in controlling for the familywise error rate (Jaccard & Guilamo-Ramos, 2002). Even though scores mostly were higher on the post-assessment, no statistically significant difference was evident in respect to solving arithmetic operations of fractions (Item 2), recognizing a fractional part-whole (Item 3), when finding an equivalent area (Item 4), and ordering fractional values (Item 5). Instead, the study revealed an impressive result where the post-assessment score was statistically significantly lower than pre-assessment [Z = -3.54, p < .001] for identifying a unit whole (Item 1) with effect size, r = .30. In both assessments, about 58% to 77% preservice teachers did not conceptually understand unit wholes—they were not able to recognize that the shaded part of two pizzas 5 8 eaten was (item 1). Instead, they believed that the two pizzas were a separated unit whole, supported Lamon’s (1999) and Ball’s (1993) argument that many individuals tended to refer to a single item (e.g., a single pizza) as a unit (one). Similarly, the Wilcoxon test analyses were performed for multiple comparisons among items under pedagogical content knowledge for adjusted α values of .05/4 = .0125. The results indicated that the select participants gained much input from the 3-week fraction instruction as measured by their pedagogical content knowledge with all items showing statistically significant differences between the assessment results (p ≤ .001). Specifically, the preservice teachers indicated statistically significantly 3 greater improvement when recognizing the accurate representation, 2 × 4 for Item 2 with Z = 4.33 and 3 an effect size, r = .36. When they were asked, “A recipe calls for 4 the cup of flour. How much flour is needed if the recipe is doubled?” during the pre-assessment, many mentioned 2 × 3 4 and 3 4 × 2 were the same representations and accurate. They stated that because of the commutative property, multiplication is reciprocal thus resulting in the same final answer. Nevertheless, after the instruction, they were able to differentiate that both representations were not identical. Preservice teachers attained statistically significantly higher scores on the post-assessment for Item 1, 3, and 4 with similar effect size, r = .27. Through a closer examination of the written responses, rich information was revealed about preservice teachers’ pedagogical content knowledge on different concepts. For instance, for Item 1 on problem posing, when creating a scenario problem on pre3 4 1 8 3 4 assessment to match the division fraction ÷ = 𝑛, many confused it with ÷ 8 = 𝑛. Here are the examples of the fraction scenarios created from the same preservice teacher in the study: Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 67 There are 24 students on the class roll. Monday at school, only ¾ of the class was present. The teacher needed to divide the class into eight groups. How many students are in each group? (Pre-assessment) How many 1/8 ft long strips of ribbon can be cut from a ribbon that is ¾ ft long? (Postassessment) The responses were similar to the previous results (Item 2) that showed many preservice teachers in the present study could not differentiate the underlying fractional concepts between two arithmetic operations. To build preservice teachers’ understanding of fractions, Dr. X allocated extra time presenting and using various concrete examples for creating scenarios during class instruction. Preservice teachers spent time practicing and generating their scenario problems based on the given fraction operation. As a result, they were able to pose meaningful and accurate scenarios during postassessment. For Item 3 and 4 that represented classroom situations for pedagogical analysis, the majority of preservice teachers were having some difficulties in analyzing the contexts and in providing brief descriptions supporting their arguments. For example, only a small number of preservice teachers could identify a missing unit from the children’s responses in Item 3. For this reason, preservice teachers had difficulties deducing the size of chunks the children considered in the case of cola. The participants were unable to analyze the children’s thinking and were difficult to make inferences based on the children’s unlabeled responses. However, other participants considered students’ thinking, understanding and misunderstanding when responding to this open-ended item and focused on the number that children provided in much detail. They attempted to predict children’s thinking with some drawings, assuming a case with 12 or 24 colas, and working backward to solve the problem. Item 4 4 focused on the conversation among three children who were comparing two fractions 4 and 4 with their 8 representation. Initially, preservice teachers’ responses failed to include the vital concept of unit-whole and the role of a common unit. However, during post-assessment, they were able to point out that the children struggled to recognize which one was a more substantial value using the fraction model, failing to understand that the size of rectangles must be the same (equal unit-whole) before comparing the fractions. For instance, a preservice teacher responded: Student 2 had the right idea that if student one had made the whole rectangles the same instead of the fractional part of a whole, he would have seen that 4/4≠4/8. However, student 3 is correct in that four parts out of 4 take up the entire rectangle while four parts of 8 take up half of a rectangle, which shows two different fractions. The students definitely could have used fraction bars to make the whole shapes even, but students 2 and 3 seemed to have understood without seeing the representation. They knew that the whole was represented with four parts for one fraction and 8 for the other. They knew 68 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 the parts of the whole were being compared. Observation during Fraction Instruction From the classroom observation, most of the preservice teachers were not experiencing problems creating scenarios involving the addition and subtraction of fractions. They were able to pose scenarios and check each other for appropriate modeling and terminologies. However, when it came to multiplication and division, preservice teachers demonstrated the most considerable anxiety because they were not able to pose scenarios that fit the algorithmic operations. They struggled to understand the concept clearly, thus were not able to come up with appropriate word problems. For instance, in one of the class sessions preservice teachers were demonstrated and illustrated scenario problems for 1 1 two fraction multiplications 2 × 3 and 3 × 2. They were guided in creating meaningful scenarios to differentiate the underlying concepts between these arithmetic operations that can produce the same 2 answer, 3 but have different meanings. Many were not capable of grasping the idea for the first time and showed their frustration by expressing their anxiety and anxiousness to their group members. Then, they asked Dr. X to show other examples that would hopefully help them build their conceptual understanding. After several explanations and practices with concrete models, most of them were able to create some meaningful scenarios involving multiplication and division even though some still did not fully understand the notion. We believed more time and effort would be necessary for them to build and understand these new concepts. Overall, we found the majority of preservice teachers enjoyed and gained much knowledge from each session of their fraction instruction and the online modules. In class, they actively participated in exploring and experimenting with various manipulatives with group members or partners to build their fractional understanding. One preservice teacher stated, “I believe that manipulatives are the best learning tool for children because they are concrete objects that they can work out the math problems with”. The response supports Piaget’s (1964) argument that hands-on materials or concrete models can help younger children build mental sense and abstract ideas in mathematics. Perceptions and Attitudes towards Fractions After the semester end, sixty-six preservice teachers submitted short written responses that described their perceptions and attitudes towards learning and teaching fractions based on their involvements in the method course. Each written response was coded into meaningful units of data in the form of phrases, sentences, or paragraphs and then these data were grouped into similar emergent themes (Johnson & Christensen, 2012). Eighteen themes were developed from the coding process and were classified into three meta-themes that related to their positive, negative, or unchanged attitudes towards fractions. We categorized preservice teachers under positive attitudes meta-theme when they showed optimistic phrases, statements and words from the written essays. Otherwise, the negative Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 69 responses noted all preservice teachers’ unfavorable endorsement towards learning and teaching fractions. The results from the coding process of the essays revealed that 62 of 66 selected preservice teachers showed some positive remarks about fractions based on their experiences from participating in the activities during the method course. Nine themes with 377 units of data were formed and considered from the responses demonstrating the change in the perceptions and attitudes. Among all the themes, the course material was frequently coded (100 units of data) indicating this might be the most critical concept noted by the majority of preservice teachers. They believed the incorporation of concrete manipulatives, online modules and videos, daily examples, and scenario problems during class instruction helped their learning about fractions. They discovered meaningful experiences in exploring and practicing different approaches to learning and teaching fractions from the method course. Also, most preservice teachers indicated that they felt comfortable with fractions (71 units of data) and their understanding of fractional concepts had increased (57 units of data) as compared to the beginning of the semester. However, they firmly believed that more time, practice, and research (39 units of data) were needed to understand fractional concepts profoundly and to be able to teach fractional concepts to young children effectively. For example, one preservice teacher stated: I feel that my attitude towards fractions has changed for the better. Dr. X did a great job explaining fractions to the class in ways I could understand. She allowed us to use different manipulatives that helped in understanding fractions rather than just using a pencil and paper. Although I am no 100% confident about teaching fractions I feel so much better about it and I know that with a little more practice I will be able to teach others what I learned about fractions in Dr. X's class. Simultaneously, preservice teachers believed the instructor (23 units of data), Dr. X, facilitated the learning of fractional concepts through various teaching aids and scenario problems. Dr. X focused on building the conceptual understanding of fractions with hands-on activities that emphasized the ‘why’ instead of the ‘how’ behind addition, subtraction, multiplication, and division of fractions. Based on preservice teachers’ past experiences, many argued that they were taught fractions with rules and memorization (40 units of data) in elementary and middle school and the learning process was extremely confusing. Also, they mentioned the ‘rote learning’ of fractions through a series of steps and formulas (i.e., algorithms) without an understanding of what was going on. A participant stated, “I have known how to get the correct answer when working with fractions, but until this class I didn’t understand how I got the answer or why it was correct”. It was an unlearning and relearning process for the majority of preservice teachers during the method course with Dr. X. In addition, creating and practicing different scenarios (e.g., problem-solving problems, word problems) involving fractions with concrete models helped preservice teachers understand the concepts better. After the course, they could “see” how portions make sense in their daily lives and would be 70 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 interested in incorporating the same teaching method into their future classroom (21 units of data). For instance, one preservice teacher discussed: When we started with creating word problems I thought no big deal. However, it became 3 4 1 3 1 3 3 4 difficult to word a problem to show if it was × versus × , because the modeling was completely different even though the answer will be the same. We practiced a lot on fractions, and also though I do not feel like I am an expert in them, I feel confident enough to teach them. I may have to consult my notes from time to time, but isn't that what those are for? I would use my notes mainly in planning out my lessons. Contrary to the many positive responses, some preservice teachers had included an opposite perception throughout their essay. We found four themes that revealed their negative attitudes about the learning and teaching of fractions. Most preservice teachers still felt anxious (64 units of data) about fractions and intimidated by mathematics (14 units of data) as a subject in general. They were nervous about teaching fractions to young children (39 units of data) because the resistance towards learning fractions had been entrenched for years and it was not going to disappear overnight demonstrating their low self-esteem and confidence towards fractions. Therefore, they were still conscious and not sure how to explain the concepts in class. For them, it was difficult and challenging to fully understand fractions especially when creating scenarios for multiplication and division with correct terminologies. Also, we found several preservice teachers were overwhelmed with the methods class (14 units of data) but were able to rationalize the situation, thus tried to motivate themselves and be more positive towards the teaching and learning of fractions. Comment for a preservice teacher: My view of fractions has changed slightly. Before this class, I did not even think about how difficult fractions would be to teach. I got nervous when I heard that it is one of the most challenging subjects to teach. When we started learning about them, I did find that fractions are tough for me to understand. Therefore, I was fearful of teaching them. I think the main thing I am concerned about is the ordering of fractions. For some reason, that gives me the most difficulty. I do believe that after this course, it has gotten a little easier to understand because of the hands-on activities that we did daily. However, I think I will always have fear when teaching fractions. The analysis of essay questions revealed two preservice teachers had entirely negative attitudes after instruction. They realized how complex working with fractions can be and hated fractions because they did not feel adequate in their teaching abilities even after taking a method course. Also, we noticed a preservice teacher claimed that she did not see any changes in her attitudes towards fractions after the instruction. She had been struggling with fractions and how to understand the concept before and after the methods course. She mentioned “It is hard for me to learn in a group setting. I am much better at Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 71 learning things one on one when I cannot comprehend them". The external generalization of the present study is limited to all elementary preservice teachers from a particular university where participants are conveniently drawn and are based on the results taken from the validated measures. However, we emphatically trust the outcomes because we incorporated both quantitative and qualitative research based on a pragmatist approach providing a perfect combination yielding “complementary strengths and nonoverlapping weaknesses” (Johnson & Turner, 2003, p. 299). Notably, the use of both closed and open-ended items uncovered several key findings from multiple perspectives that add to the existing literature on the preservice teachers’ level of knowledge and attitudes towards fractions. Results of the present study advance current understanding of the effect from a 3-week fraction instruction of a mathematics method course in several valuable ways. First, it provides useful insights into the potential benefits of integrating concrete models, problem-solving, and problem-posing activities on building prospective teachers’ profound knowledge of teaching fractions. Second, the study was distinctive because it represents the first study using a mixed-methods research design to more deeply understand the complex phenomena of selected preservice teachers’ content knowledge and pedagogical content knowledge of fractions, and attitudes towards fractions. Seventy-one preservice teachers participated in the study where they complete pre- and postassessments on content knowledge and pedagogical content knowledge of fractions. Also, they submitted an essay about attitudes towards fractions after the 3-week instructional unit. Importantly, the study showed that the incorporation of problem-solving, problem posing, and hands-on activities have the potential to assist instructors in making fraction instruction more meaningful and enjoyable thus eventually helping learners to develop conceptual knowledge of fractions (Thompson, 1994). Some other significant outcomes and issues need to be addressed aside from the inconclusive results as measured by the closed and open assessment tasks. Related to the select preservice teachers’ content knowledge of fractions, results from the descriptive statistics revealed low percentages of preservice teachers achieved the full points for most of the items on both pre-and post-assessments. The results might indicate the participants were weak on the particular fraction concepts being tested. Specifically, we noticed the select preservice teachers were not aware that the unit-whole may include more than one item or may consist of items packaged as one known as a composite unit. The behavior was consistently evident when they were identifying the unit-whole of Item 4 (closed-ended) and Item 1 (open-ended). On pre- and post-assessments, almost 90% of preservice teachers were not able to notice that children were considering a different set of unitwhole on Item 4 (close-ended). Similarly, the picture for Item 1 (open-ended) showed two pizzas with the same or different kinds, which is called a one two-unit and not two one-unit pizzas. However, many 1 preservice teachers assumed a single object (i.e., a single pizza) as a unit (one) and answered 1 4 pizza 5 instead of 8 pizza. Lamon (1999) argued that the concept of the unit has frequently been neglected in 72 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 classroom discussions. Majority teachers and textbook authors emphasized using the same unit-whole such as one cake in which making students assumed a unit was always a single object (Lamon, 1999). Lamon argued many classroom instructions failed to introduce the vital concept of unit-whole of fractions to students. CONCLUSION While this study revealed the preservice teachers’ lack of content knowledge of fractions, the results showed compelling evidence of their understanding of teaching fractions supported previous research by Ball (1993), Hill (2010), Ma (1999), Newton (2008), and Timmerman (2004). It is noteworthy to see that they demonstrated statistically significant improvement in their level of pedagogical content knowledge even though they received only a three-week fraction instruction unit mainly focusing on problem posing and hands-on activities. We believe the close interaction with the instructor and quality of instruction during the mathematics method course possibly helped preservice teachers develop knowledge about pedagogy and nurture their fraction teaching repertoire (White et al., 2005). Our data warrant further investigation into the relationship among the duration of instruction, the role of instructor, quality of instruction, and level of fractional knowledge they gained. Therefore, a more thorough examination is critical to determine the factors that might affect the level of knowledge before and after instruction. In the post-assessment, many preservice teachers successfully generated and identified meaningful and accurate fraction situations that matched the given arithmetic operations. When reviewing their fraction essays, they mentioned how problem posing and hands-on activities had an impact on their learning and understanding of fractions. The majority of preservice teachers showed positive attitudes towards fractions after receiving the instruction and believed it was an eye-opening session. Still, some expressed their concerns about additional practice and the time needed for developing confidence in teaching fractions. Our results revealed that the select preservice teachers were able to reflect on students’ work and to suggest basic teaching approaches for helping students’ misunderstanding of fraction concepts. However, the responses showed they had limited practical experiences teaching fractions to children. Because the participants would be elementary school teachers later, more exposure working directly with students in a real classroom setting is needed. The experience responding to elementary students would develop their pedagogical skills and improve preservice teachers’ understanding of educational components (Llinares, 2002; White et al., 2005). Taken together, teacher education services are a place for teachers to build expertise, teach skills, values, and understanding in order to become effective teachers in mathematics (Llinares, 2002). Nonetheless, the prior knowledge and behaviors of service teachers have a significant impact on what and how they learn during their teacher training programs (Llinares, 2002). Teacher educators are responsible for providing potential teachers with the in-depth knowledge and experience required to Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 73 effectively teach the basic concepts of fractions. ACKNOWLEDGMENTS The author would like to acknowledge the National University of Malaysia through the GG-2019015 and GGPM-2013-085 grants for providing support for the research work. The open-access publishing fees for this article have been partially covered by the Texas A&M University Open Access to Knowledge Fund (OAKFund), supported by the University Libraries and the Office of the Vice President for Research. REFERENCES Ball, D. L. (1993). Halves, pieces, and twoths: Constructing and using representational contexts in teaching fractions. In Carpenter, T. P., Fennema, E., Romberg, T. A. (Eds.). Rational numbers: An integration of research (pp.157-195). Hillsdale, NJ: Erlbaum. Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389-407. http://dx.doi.org/10.1177/0022487108324554. Barnett-Clarke, C., Fisher, W., Marks, R., & Ross, S. (2010). Developing essential understanding of rational number: Grades 3-5. Reston, VA: The National Council of Teachers of Mathematics. Collins, K. M. T., Onwuegbuzie, A. J., & Jiao, Q. G. (2007). A mixed methods investigation of mixed methods sampling designs in social and health science research. Journal of Mixed Methods Research, 1(3), 267-294. http://dx.doi.org/10.1177/1558689807299526. Collins, K. M. T., Onwuegbuzie, A. J., & Sutton, I. L. (2006). A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4(1), 67-100. Cramer, K., & Wyberg, T. (2009). Efficacy of different concrete models for teaching the part-whole construct for fractions. Mathematical Thinking and Learning, 11(4), 226-257. http://dx.doi.org/10.1080/10986060903246479. Erlandson, D. A., Harris, E. L., Skipper, B. L & Allen, S. D. (1993). Doing naturalistic inquiry: A guide to methods. Newbury Park, CA: Sage. Empson, S. B. (2002). Organizing diversity in early fraction thinking. In B. Litwiller & G. Bright (Eds.), Making sense of fractions, ratios and proportions: 2002 Yearbook (pp. 29-40). Reston, VA: National Council of Teachers of Mathematics. Glaser, B. (1965). The constant comparative method of qualitative analysis. Social Problems, 12(4), 436-445. http://dx.doi.org/10.2307/798843. Hill, H. C. (2010). The nature and predictors of elementary teachers’ mathematical knowledge for teaching. Journal for Research in Mathematics Education. 41(5), 513-545. IBM Corp. (2013). IBM SPSS Statistics for Windows, Version 21.0 [Computer software]. Armonk, NY: IBM Corp.). Jaccard, J., & Guilamo–Ramos, V. (2002). Analysis of variance frameworks in clinical child and adolescent psychology: Issues and recommendations. Journal of Clinical Child and Adolescent Psychology, 31(1), 130 –146. http://dx.doi.org/10.1207/S15374424JCCP3101_15. 74 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 Johnson, B., & Christensen, L. (2012). Educational research: Quantitative, qualitative, and mixed approaches (4th ed.). Boston, MA: Sage. Johnson, R. B., & Turner, L. A. (2003). Data collection strategies in mixed methods research. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 297-319). Thousand Oaks, CA: Sage. Kane, R., Sandretto, S., & Heath, C. (2002). Telling half the story: A critical review of research on the teaching beliefs and practices of university academics. Review of Educational Research, 72(2), 177–228. http://dx.doi.org/10.3102/00346543072002177. Lamon, S. J. (1999). Teaching fractions and ratios for understanding: Essential content knowledge and instructional strategies for teachers. Mahwah, NJ: Erlbaum. Learning Mathematics for Teaching. (2008). Mathematical knowledge for teaching measures: Rational number. Ann Arbor, MI: Author. Lin, C. (2010). Web-based instruction on preservice teachers’ knowledge of fraction operations. School Science and Mathematics, 110(2), 59-70. http://dx.doi.org/10.1111/j.1949-8594.2009.00010.x. Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Newbury Park, CA: Sage. Llinares, S. (2002). Participation and reification in learning to teach. The role of knowledge and beliefs. In G. Leder, E. Pehkonen, & G. Tö rner (Eds.), Beliefs: A hidden variable in mathematics education (pp. 195–210). Dordrecht, The Netherlands: Kluwer. Ma, L. (1999). Knowing and teaching elementary mathematics: Teachers’ understanding of fundamental mathematics in China and the United States. Mahwah, NJ: Erlbaum. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author. Newton, K. J. (2008). An extensive analysis of preservice elementary teachers’ knowledge of fractions. American Educational Research Journal, 45(4), 1080-1110. http://dx.doi.org/10.3102/0002831208320851. Onwuegbuzie, A. J. (2003). Expanding the framework of internal and external validity in quantitative research. Research in the Schools, 10(1), 71-90. Onwuegbuzie, A. J., Johnson, R. B., & Collins, K. M. T. (2009). A call for mixed analysis: A philosophical framework for combining qualitative and quantitative. International Journal of Multiple Research Approaches, 3(2), 114-139. http://dx.doi.org/10.3102/000283120832085110.5172/mra.3.2.114. Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing data in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 351–383). Thousand Oaks, CA: Sage. Osana, H. P., & Royea, D. A. (2011). Obstacles and challenges in preservice teachers’ explorations with fractions: A view from a small-scale intervention study. Journal of Mathematical Behavior, 30(4), 333-352. http://dx.doi.org/10.1016/j.jmathb.2011.07.001. Piaget, J. (1964). Part I: Cognitive development in children: Piaget development and learning. Journal of Research in Science Teaching, 2(3), 176-186. http://dx.doi.org/10.1002/tea.3660020306 Provalis Research. (2009). QDA Miner 3.2. User‘s guide. Montreal, QC, Canada: Author. Rosli, Goldsby, Onwuegbuzie, Capraro, Capraro, & Gonzalez, Elementary preservice teachers’ knowledge … 75 Sarama, J., & Clements, D. H. (2009). "Concrete" computer manipulatives in mathematics education. Child Development Perspectives, 3(3), 145-150. http://dx.doi.org/10.1111/j.17508606.2009.00095.x. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14. http://dx.doi.org/10.3102/0013189X015002004. Sowder, J. T., Philipp, R. A., Armstrong, B. E., & Schappelle, B. P. (1998). Middle-grade teachers’ mathematical knowledge and its relationship to instruction: A research monograph. New York, NY: State University of New York. Teddlie, C., & Tashakkori, A. (2006). A general typology of research designs featuring mixed methods. Research in the Schools, 13(1), 12-28. Thompson, P. W. (1994). Concrete materials and teaching for mathematical understanding. Arithmetic Teacher, 41(9), 556-558. Timmerman, M. (2004). The influences of three interventions on prospective elementary teachers’ beliefs about the knowledge base needed for teaching mathematics. School Science and Mathematics, 104(8), 369-382. http://dx.doi.org/10.1111/j.1949-8594.2004.tb18003.x. White, A. L., Way, J., Perry, B., & Southwell, B. (2005). Mathematical attitudes, beliefs and achievement in primary pre-service mathematics teacher education. Mathematics Teacher Education and Development, 7, 33-52. 76 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 59-76 ISSN 2087-8885 E-ISSN 2407-0610 Journal on Mathematics Education Volume 11, No. 1, January 2020, pp. 17-44 PROSPECTIVE MATHEMATICS TEACHERS’ COGNITIVE COMPETENCIES ON REALISTIC MATHEMATICS EDUCATION Rezan Yilmaz Ondokuz Mayis University, Faculty of Education, Samsun, Turkey Email: [email protected] Abstract Realistic Mathematics Education (RME) is based on the idea that mathematics is a human activity; and its main principle is to ensure the transition from informal knowledge to formal knowledge through contextual problems. This study aims at revealing how RME is configured in the minds of prospective mathematics teachers and their cognitive competency in that sense. For that purpose, at the end of the process, in which the approaches used in mathematical education including RME are examined and interpreted, 32 prospective teachers were asked various open-ended questions. Moreover, they were expected to pose contextual problems that could be used in RME. After analysing the obtained data via qualitative research techniques, it is seen that the majority of the prospective teachers possesses theoretical knowledge on RME. However, it is also observed that their ability to present its differences and similarities with other approaches and to pose contextual problems suitable to RME has been decreased. Keywords: Realistic Mathematics Education, Prospective Mathematics Teacher, Cognitive Competency, Contextual Problem Abstrak Pendidikan Matematika Realistis (PMR) didasarkan pada gagasan bahwa matematika adalah aktivitas manusia; dan prinsip utamanya adalah memastikan transisi dari pengetahuan informal ke pengetahuan formal melalui masalah kontekstual. Penelitian ini bertujuan untuk mengungkapkan bagaimana PMR dikonfigurasi dalam pikiran calon guru matematika dan kompetensi kognitif mereka dalam pengertian itu. Untuk tujuan itu, pada akhir proses, 32 calon guru ditanyai berbagai pertanyaan terbuka, yang mana pendekatan yang digunakan dalam pendidikan matematika termasuk PMR diujikan dan ditafsirkan. Selain itu, mereka diharapkan untuk dapat mengemukakan masalah kontekstual yang dapat digunakan dalam pendekatan PMR. Setelah menganalisis data yang diperoleh melalui teknik penelitian kualitatif; terlihat bahwa mayoritas calon guru memiliki pengetahuan teoritis tentang PMR. Namun, pengamatan lebih jauh terkait kemampuan mereka untuk menyajikan perbedaan dan persamaan antara pendekatan PMR dengan pendekatan yang lain dan pengajuan masalah kontekstual yang sesuai dengan PMR, telah berkurang. Kata kunci: Pendidikan Matematika Realistik, Calon Guru Matematika, Kompetensi Kognitif, Masalah Kontekstual How to Cite: Yilmaz, R. (2020). Prospective mathematics teachers’ cognitive competencies on realistic mathematics education. Journal on Mathematics Education, 11(1), 17-44. http://doi.org/10.22342/jme.11.1.8690.17-44. The studies conducted in mathematical education, which is highly impacted by theoretical and methodological framework of psychology (Sriraman & English, 2010), have paved the way for the innovative ideas on learning including new learning outcomes, new types of learning processes, and new instructional methods that are both wanted by society and currently stressed on in psychological and educational theory (Simons, Van der Linden, & Duffy, 2000; Yackel, Gravemeijer, & Sfard, 2011). These ideas occur in a manner that is totally different from behavioral approach studies (Aubrey & Riley, 2016) which traditional education methods are based on. And their approaches can be fundamentally categorized as cognitive approaches and constructivism. Cognitive theories focus on 17 18 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 the conceptualization of one’s learning process and emphasize how information is received, organized, stored and retrieved in one’s mind (Davis, 1990; English, 1995; Ertmer & Newby, 1993; Jonassen, 1991; OECD, 2003). Despite being viewed as a branch of cognitivism as a result of expressing learning as a cognitive activity (Ertmer & Newby, 1993), constructivism is based on the idea that the mind filters the input received from life in order to create its own reality (Jonassen, 1991a) and the information is not simply received but actively constructed by the person himself/herself (Driscoll, 2005). The different perspectives in education led to radical changes in objectives and nature of mathematics as well as many other areas. This change in mathematics education particularly emphasizes that mathematics learning can occur only when students discover things through actual experience and structured problem-solving procedures and finally by means of interaction process among students and/or teachers (Kwon, 2002). In addition to this, many countries aiming at raising productive and innovative individuals in the 21st century made radical changes in their curriculum with the purpose of attaching much more importance to the quality of their mathematics education. Essentially, various countries such as UK, USA, Singapore, Finland and Australia believe that mathematics enables the improvement of thinking capacity as well as its use in real life in a critical, creative and logical manner. These countries have begun to pursue the goal of raising individuals with problem solving, reasoning, and connecting skills, as well as productive disposition and conceptual comprehension (Cai & Howson, 2013; Stacey, 2005). It is believed that among these skills, the problem solving is the key skill in the 21st century (Barell, 2010). Moreover, it is asserted that the teaching process can be implemented for students through case-based or problem-based learning methods by presenting authentic problem situations called as real-world problem (Barell, 2010; Barrows, 1986). Conceptual comprehension, on the other hand, is a process that takes place in coordination with such skills (Wu, 1999) and involves the understanding of concept, relation and operation (National Research Council, 2001). This new perspective shares many common points with the theoretical perspective of Realistic Mathematics Education (RME) (Bray & Tangney, 2016; Kwon, 2002) as discussed by Freudenthal (1973). RME focuses on mathematization that is actualized through the re-invention of formal mathematics. In RME, the student is informally guided by the teacher in a class-interaction, thus is encouraged to utilize self-developed models in order to solve and interpret the experientially real contextual problems (Dawkins, 2015; Gravemeijer, 1994, 1999; Treffers, 1991). In this regard, it is believed that RME contributes to the formation of the targeted skills such as problem solving, reasoning and connecting; as well as to conceptual comprehension (Borko & Putnam, 1996; Hadi, 2002). Teachers’ Pedagogical Knowledge (PK), which includes knowledge the teachers have on curriculum and teaching methodology as well as on how to teach them (Lianghuo, 2014), and Pedagogical Content Knowledge (PCK) that is referred to as “special amalgam of content and Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 19 pedagogy” (Shulman, 1987), are important determinants of instructional quality that impact students’ learning gains and motivational development (Baumert & Kunter, 2013). And also they have vital importance in gaining the targeted skills (Borko & Putnam, 1996). This situation causes many results, which could affect teacher education and causes major changes in prospective teachers' knowledge (Battista, 1994; Putnam & Borko, 2000; Szydlik, Szydlik, & Benson, 2003). However, teachers’ efforts to change their PK and practices are usually not reflected on their tendencies at skill-based applications in the targeted curriculum (Anderson, White, & Wong, 2012). Although, it is difficult to evaluate teachers’ PCK (Beswick & Goos, 2012), understanding the pedagogical theories underlying the radical changes about mathematics education and detecting effective teaching strategies are significant in terms of being able to professionally assessing teachers (Sowder, 2007). Furthermore, besides prospective teachers' knowledge of general and specific approaches; what is essentially important is whether they comprehend when, where, how, and why they should use these approaches, rather than their variety (Feiman-Nemser, 2001; Tsamir, 2008). However, many prospective teachers are not ready to use them the way they should (Herman & Gomez, 2009). Therefore, it is believed that it is important to know to what extent prospective teachers have cognitive competences about these approaches, in order to be able to choose the approach that will be used depending on place and time and to implement them in a manner that is suitable for the purpose. In a general sense, the study aims at identifying prospective teachers' cognitive competency regarding the approaches before moving onto practice from theory. In this regard, the study mainly focuses on RME in terms of the approach and the answer to the question of "How are the cognitive competencies of prospective mathematics teachers related to RME?" has been sought. Thus, the subproblems of "How do prospective mathematics teachers explain RME and its implementation? How do they interpret the similarities and differences with other approaches? How a contextual problem do they pose in compliance with the theoretical structure of RME?" have been discussed. In the below-mentioned theoretical framework, firstly the definition and primary elements of RME have been explained, and information related to its implementation has been presented. Subsequently, the criteria used in determining the cognitive competency and the levels of cognitive competency have also been explained. THEORETICAL FRAMEWORK Realistic Mathematics Education (RME) RME, which is based on the idea that mathematics is a human activity (Freudenthal, 1973) and the idea that student achieves formal mathematics knowledge by using his/her informal knowledge by means of re-inventing under the guidance of a teacher (Treffers, 1991) has a significant place in the studies conducted in the field of mathematical education (e.g. Barnes, 2004; Beswick, 2011; Bray & Tangney, 2016; Makonye, 2014; Rasmussen & King, 2000; Streefland, 1991). In this approach, the informal knowledge in real life is transformed into formal knowledge after being abstracted and is 20 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 again associated with real life as a result of mathematical implementations (De Lange, 1996). Transformation is believed to be achieved by contextual problems that are experientially real to the students (Gravemeijer, 1999). The process of conceptual mathematization in RME has been given in Figure 1. In this process, mathematical concepts start to develop from the real world and ends with the reflection of the solution back to the real world. Figure 1. Conceptual mathematization (De Lange, 1999) The contextual problems used in RME are mathematical problems presented in the real life situations that children are familiar with, through stories that are fictionalized from the real world. These problems can be a word problem, a game, a drawing, a newspaper clipping, a graph or the combinations of such elements. At the same time, a pure mathematical problem can also be a contextual problem. However, the main point here is to what extend the problem would fulfill the criteria of being experientially real or authentic; and thus would provide a concrete orientation towards a new concept/skill; and would also allow utilization of prior knowledge (De Corte, 1995; Doorman, Drijvers, Dekker, van den Heuvel-Panhuizen, de Lange, & Wijers; 2007; Gravemeijer, 1999). Gravemeijer (1994, 2001) emphasizes the necessity for three main elements while designing education in RME, which includes the following: 1. Guided reinvention through progressive mathematization 2. Didactical phenomenology 3. Self-developed or emergent models Guided reinvention is based on configuring and organizing problems in order to discover rules by revealing mathematical factors in a problem. This research, which is conducted with a strong intuitional component, is considered to be the discovery or reinvention of mathematical conception (De Lange, 1987; Freudenthal, 1973, 1991). In this process, the teacher should design the roadmap to enable students to learn correctly and should provide the students with the opportunity to experience a process that is similar to the discovery process of mathematicians (Gravemeijer, 1994; 2001). By doing so, students have the opportunity to obtain knowledge by themselves (Freudenthal, 1991; Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 21 Gravemeijer, 1999; Yackel & Cobb, 1996) and they mathematize the contextual problems by solving them (Treffers, 1987; 1991). Guided reinvention through progressive mathematization can be considered as a two-stage process: horizontal and vertical mathematization. Horizontal mathematization is where one uses informal strategies by schematization in order to define and solve contextual problems, in other words transforming real life problems into mathematical problems. Vertical mathematization is to abstract the conception in the world of symbols and solve the problem by adopting different models or find the relevant algorithm by using mathematical language in the light of informal strategies (Freudenthal, 1991; Gravemeijer, 1994; Treffers, 1987, 1991; Van den Heuvel-Panhuzen, 2003). In Figure 2, horizontal and vertical mathematization is described. Figure 2. Horizontal and vertical mathematizations (adapted from Gravemeijer, (1994)) Didactical phenomenology requires working with phenomenon that are meaningful to students in the process of learning mathematics, can be organized by students, are stimulating for the learning process and meet four functions including concept formation, model formation, applicability and practice (Gravemeijer, 1994, 2001; Treffers & Goffree, 1985). Self-developed or emergent model bridges informal knowledge of students with formal knowledge while solving a problem. At the beginning, the student develops a model, which gradually becomes a dynamic and holistic model compatible with his or her own mathematical thinking after generalizing and formalizing processes (Gravemeijer, 2001; Treffers, 1991). Thereby, at the end of this process, which is named as the transformation from 'model-of' to 'model-for', the student obtains a model that enables him/her to achieve mathematical reasoning (Gravemeijer, 1999). Cognitive Competency Cognition refers to the variables with respect to the kind and quantity of information, and the classification of relations among the variables of information (Kraiger, Ford, & Salas, 1993). 22 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 Cognitive domain in learning contains learnings, in which person's mental sides are in the foreground (Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956; Bloom, 1994). Cognitive competency is a psychological construction which cannot be directly observed but can be inferred from the behavior or performance of an individual on content-relevant tasks (Wang, 1990). Researchers have asserted various taxonomies in order to evaluate this kind of learnings. Some of these taxonomies are Bloom’s Taxonomy (Bloom, et al. 1956), Revised Bloom Taxonomy (Anderson & Krathwohl, 2001), Barrett’s Taxonomy (Barrett, 1976), Pearson-Johnson Taxonomy (Pearson & Johnson, 1978), Webb’s Depth of Knowledge Levels (Webb, 1997, 1999) and Program for International Student Assessment (PISA)’s Competency Levels (OECD, 2003). Bloom’s Taxonomy among such taxonomies, is the first and most widely accepted classification (Granello, 1995) in the subject of cognitive abilities and educational objectives used in education. In a similar way, PISA is implemented and reported in many countries and regarded as a new approach in national and international evaluation (Sadler & Zeidler, 2009). Categories are classified in Bloom Taxonomy from simple to complex and from concrete to abstract acting as prerequisite for one another: Knowledge refers to a person's acts of remembering such as recognizing, expressing when asked, or repeating the characteristics of any object or event from his/her memory; Comprehension means interpreting, assimilating, and expressing the obtained targets without losing their meanings at knowledge level; Application, is when a person implements knowledge by solving the problem in a new situation by making use of his/her learning at knowledge and comprehension level; Analysis refers to cognitively differentiating among the items of a pattern or knowledge in terms of their relationships and organizations; Synthesis means bringing together and creating a whole of the items in such a way that they would bare characteristics such as innovation, originality, and creativity based on certain relationships and rules. Evaluation is when a person decides whether the products created are competent enough by stating justifications (Bloom, et al. 1956; Bloom, 1994; Krathwohl, Bloom, & Bertram, 1973; Krathwohl, 2002). In PISA, OECD (2003) has determined three levels for detecting the competency levels of students in order to define their cognitive activities. These levels include reproduction, connection, and reflection. In reproduction level, already known contents, previously used knowledge, standard algorithms, and elementary formula are used and basic operations are conducted. In connection level, less commonly known contents are interpreted and explained; systems, representation of which are different, are obtained through association; and necessary strategies are chosen and used for extraordinary problem solving. In reflection level, in which comprehension is required; reflection, creativity, and knowledge necessary for solving complex problems are associated; observed results are generalized and justified; and abstraction is carried out. The association between Bloom’s Taxonomy and PISA’s Competency Levels is indicated in Figure 3. Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 23 Figure 3. The relations of cognitive competency (OECD, 2003; Bloom, et al. 1956) METHOD This research is a qualitative study that has been conducted with the purpose of researching the cognitive competencies of prospective mathematics teachers related to the approaches used in mathematics education. Research design is a case study, which has been evaluated as a case of RME. This study has been conducted in the faculty of education of a state university in the Black Sea region of Turkey with 32 prospective mathematics teachers, 20 of which are female and 12 of which are male. These prospective mathematics teachers are all senior students studying their fourth year of their 5-year education plan. The majority of these prospective mathematics teachers have successfully completed fundamental education courses such as introduction to education, educational psychology, guidance, theories and approaches of learning and teaching, curriculum development and instruction besides pure mathematical courses. All participants were informed about the process of the research. They volunteered to attend the research and gave the researcher the permission to use the data acquired from their interpretations and their posed problems in the manuscript. The prospective mathematics teachers had been taking Methods of Teaching Mathematics lesson during the time this study was conducted. In the lesson instructed by the researcher and lasted for four hours a week, some approaches about mathematical education (e.g. Ausubel’s meaningful learning approach (Ausubel, 1963), Freudenthal’s RME approach (Freudenthal, 1973), Bruner’s discovery learning approach (Bruner, 1961) have been elaborately examined and their implementation in teaching has been interpreted and discussed. In this research, it has been aimed to reveal cognitive status of prospective teachers about RME at the end of their 12-hour experience. In this study, two sessions that were suggested by Selter (1997, 2001) and grounded on the study of Zulkardi (2002) about how preservice teachers have developed the RME learning environment, were taken into consideration: understanding the new approach by providing a theoretical overview and by actually doing mathematics-the mathematical component, designing 24 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 instructional materials-the didactical component. Thus, cognitive competencies of prospective teachers in the theoretical sense were focused on and the components of the sessions were discussed as “overview of RME theory, doing mathematics and designing contextual problems” (Zulkardi, 2002). During ‘overview of RME theory’ session, firstly, the prospective teachers were provided with theoretical information on RME, and then they conducted activities about the approach. Afterwards, the implementation process and effectiveness of the activities were discussed. The prospective teachers stated their comments on the elements of the transition of RME application from real life to mathematics such as the characteristics of contextual problems applicable to horizontal mathematization, the modelings created during its solution, and roles of teacher and students. Then, they continued to discuss about vertical mathematization. During doing mathematics session, prospective teachers were treated as learners while the researcher performed as a teacher and it was aimed that they learn how to teach using with RME. In Appendix, there are some examples of the contextual problems used in the applied activities during these sessions (Altun, 2011; Fauzan, 2002; Feijs, 2005; Wubbels, Korthagen, & Broekman, 1997). During the second session, for instance, Feijs’s (2005) Grand Canyon Problem (problem 4 in appendix) developed in line with the construction of RME learning environment was implemented with prospective teachers. The prospective teachers formed 3- person student groups in real classroom environment and two of the prospective teachers from the group sat on one desk, whereas the other one sat on the desk that is parallel to the other desk. On the paper, which was hung down these parallel desks in order to create an imaginary river in the gap between the two desks, the points where the vision lines of these two people have been drawn with the help of a third person and canyon tables activity have been implemented. In the practice process, discussions were held on how they identify situations that can be seen or not seen by a person and how they perceive the vision lines from these situations. After the implementation, the situation of treating steepness of vision line as phenomena was examined according to the didactical phenomenology criterion where the main idea is familiarity and appeal for them. At this point, the mathematical content, which requires the ratio of a right triangle formed by an angle, was associated with the concept of steepness, and the state of abstraction resulting in the concept of tangent was also discussed. In addition, the self-developed or emergent models’ criterion of the approach was also discussed, considering the models that they can suggest for solution of the problem. During designing contextual problem session, it was aimed to relate the context and the concept in a learning environment. In this manner, prospective teachers could learn how to design contextual problems to use in RME. So, they were asked to pose a contextual problem, for which they were given an adequate period of time and which would enable them to conduct horizontal mathematization. Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 25 After these sessions, they were asked some open-ended questions about RME and the problems they posed; moreover, they were expected to write down their opinions about these questions. The questions asked to the prospective teachers are as follows: 1. What is RME? Please explain. 2. How is RME applied? How is this approach different from the other approaches you have learned? Please explain. 3. Pose a (contextual) problem that you could apply in RME, and explain for which conception's mathematization, this problem will be used. Please, evaluate the approach, on which the problem you posed will be used, by considering different approaches. The written answers received from the prospective teachers and their contextual problems were analyzed, through the descriptive analysis method so as to determine their cognitive competencies about RME. In descriptive analysis, primarily a framework is established for analysis; the data are processed in accordance with thematic framework; findings are described and interpreted (Patton, 1990). Due to that reason; PISA Competency Clusters (OECD, 2003) and Cognitive Domain Taxonomy (Bloom, et al. 1956), which are displayed in Figure 3 below in order to reveal the cognitive competencies of prospective mathematics teachers, were used as the foundation for the data analysis. In this process, the relevant categories and data related to sub-categories that belong to each category have been processed and common themes were determined (Creswell, 1998; Patton, 1990). In the process of analyzing the data, the data collected from the first part of the question asked to the prospective teachers have been processed in the reproduction-knowledge category; the data collected from the second part of the first question have been processed in the reproduction-comprehension category; the data obtained from the first part of the second question have been processed in the connection-application category; the data collected from the second part of the second question have been processed in the connection-analysis category; the data retrieved from the first part of the third question have been processed in the reflection-synthesis category; and the data collected from the second part of the third question have been processed in the reflection-evaluation category. Furthermore, the sub-categories within each category have been composed as adequate ones in and deficient (or incorrect) ones in the related category. In order to ensure the reliability of the study, two researchers, who have completed their PhD in the field of mathematics education and are experts in qualitative studies, firstly individually analyzed the data then it was discussed among the members until they reached a consensus on overall categories, sub-categories and themes. So, the use of multiple experts, as well as the use of their evaluations has led to conformability of the data. The credibility has been increased through the data obtained from the contextual problems consisting of the written answers given by the participants to the open-ended questions. For transferability, in order to help applying the findings to the other contexts, the description of the context was delivered in a clear and detailed manner. In other words, 26 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 the participants, the approaches examined and the activities conducted in the classroom were elaborated. Besides, in order to ensure that the results can be conveyed into similar media, the obtained findings have been supported with the quotations and detailed descriptions have been made (Berg, 2001; Lincoln & Guba, 1985; Yıldırım & Şimşek, 2005). RESULTS AND DISCUSSION The contextual problems and answers given by 32 prospective teachers participating in the study were examined and evaluated; furthermore, common themes were created based on their cognitive competencies. The categories and sub-categories created for determining common themes are displayed in Table 1. Moreover, frequencies of categories and sub-categories are also demonstrated in that table. The findings on cognitive competencies of the prospective teachers have been explained within each category by directly quoting them. Table 1. Frequencies and percentages of categories and sub-categories of prospective teachers' competencies Category Knowledge Reproduction Sub-category Having knowledge about RME 87.5 4 12.5 27 84.375 5 15.625 26 81.25 6 18.75 18 56.25 Deficient or incorrect analysis of RME 14 Posing an original (or an unfamiliar) contextual 11 problem Deficient or incorrect posing about contextual 21 problem 43.75 Deficient or incorrect knowledge about RME Applying knowledge about RME Deficient or incorrect application about RME Connection Analysis Synthesis Reflection Evaluation % 28 Making explanation or interpretation about RME Comprehension Deficient or incorrect comprehension about RME Application Frequency Analyzing knowledge about RME Making valuation by passing a judgment No answer or deficient evaluation 34.375 65.625 5 15.625 27 84.375 Reproduction Knowledge In the first part of the 1st question, prospective teachers were asked to answer to define RME approach. Thereby, it was aimed to reveal the level of knowledge of the prospective teachers. The 28 (%-87.5) of 32 prospective teachers have correctly answered to the question 'what is RME?'. An Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 27 example from the answers given by the participants is stated below: “We come across with mathematics in many parts of our everyday lives and we have to do the math. RME starts with this idea. In RME, a problem about real life are presented to students before teaching them the subject, and students solve this problem by making use of their previous knowledge through their own models. Thus, the student achieves a mathematical expression” (3rd PT). It was observed that 4 (12.5%) of the participants have deficiently or incorrectly defined RME. There were no participants, who did not answer to the question. One example from one of the prospective teachers, who deficiently answered to the question, is as follows: “Enabling them comprehend mathematics by giving them examples from life. How can we make life easier by using mathematics in our life. This is based on this” (13th PT). The prospective teacher, who deficiently answered the question, mentioned the association of the approach with real life and said that mathematics could be learned through real life examples. However, the fact that she did not mention contextual problems, which are considered to be necessary for the transition from real life to mathematics and is the focus of the approach, demonstrates that the prospective teacher does not have adequate knowledge about the approach. The answer received from one of the prospective teachers, who incorrectly answered the question, is as follows: “RME is an approach that could help us solve the problems we may face in life. It is teaching students a subject by giving examples of the events we may experience in our lives about the subject” (31th PT). Even though the prospective teacher talks about real life problems, his/her '...teaching students a subject by giving examples of events from life...' statement indicates that he/she has inaccurate knowledge on the roles of the student and teacher and also on the application of the approach. Recalling the knowledge is vital in terms of making learning meaningful and solving a problem, thereby this knowledge can be used in more complex assignments (Anderson & Krathwohl, 2001). Moreover, Ayer (1936) discusses that when something is defined, it should contain the definition itself or its synonymous or expressions with equivalent meanings. Examining the answers received by the prospective teachers; majority of the prospective teachers have knowledge on RME. Comprehension In the 1st question, which was asked in order to understand whether the prospective teachers comprehended RME or not, they were expected to explain the approach. Their explanations suggest that 27 (84.375%) of the participants have correctly comprehended RME. One of the teachers, who correctly comprehended RME, explained it as the following: 28 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 “The mathematical conception, which is intended to be taught to the student, is taught through a problem that could be experienced in real life. The student tries to solve this problem, which is created by preparing the necessary setting, through his/her own knowledge and modelings. In a sense, how mathematicians discover mathematics in the events, which they come across with in real life, is experienced by the student. They are introduced to mathematical knowledge by solving the problem. Thus, this problem should be compatible with real life and should enable students reach mathematical conceptions” (24th PT). The explanations suggest that 5 (15.625%) of the prospective teachers didn’t comprehend RME. One of these prospective teachers (16th PT) could not adequately comprehend it even though he/she correctly defined it. The other four prospective teachers are in the category of those who incorrectly or deficiently answered the question at the knowledge level. The prospective teacher, who could not comprehend the conception, explained it as the following: “In this approach, real life problems are asked to the student. We solve the problem and teach them the subject by saying this problem is its application in mathematics” (16th PT). The statements of the prospective teacher, suggest that it should be begin with real life problems. According to the approach, the problem should be solved through a model by the student himself/herself by discussing it with his/her group friends. However, his/her statements about the teachers' solving the problem for students and conducting the mathematization indicate that the prospective teacher incorrectly comprehended the approach. If we consider explaining as conveying the meaning to the other person and interpretation as realizing broad understandings (OECD, 2009); it is possible to say that both actions refer to comprehension. Thus, the answers given via explanations or interpretations indicate that the majority of the prospective teachers could significantly comprehend the approach. Connection Application Examining the answers received by the prospective teachers for the first part of the 2nd question about the application of RME, 26 (81.25%) of them correctly answered to the question about the application of this approach. One of the prospective teachers explains the application of approach as follows: “Firstly, a real life problem, which could enable student to reach a conception at the end, is given to the student. The student tries to actively solve this problem through the model he/she desires by discussing. The teacher helps them as a guide. At the end, the Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 29 student reaches mathematics through the model he/she created and is introduced to the desired conceptions. This part, in other words transition from real life to mathematics is called horizontal mathematization. Afterwards, the operations conduced within mathematics are called vertical mathematization. For example, everybody solved the snake problem we applied in classroom through their own desired models and discussion; and everybody was introduced to geometrical progression without even realizing. While doing so, we discussed and made associations in the classroom; and the example was modelled through expressions such as exponential numbers or common factors. This part was horizontal mathematization. Then we continued with vertical mathematization through conducting operations about geometrical progression. What is important in the approach is that students create their own models, discover mathematics through teachers' guidance, and obtain knowledge” (18th PT). The above-quoted prospective teacher pointed out the three main factors, which are required in RME applications and stated by Gravemeijer (1994, 2001) including reinvention through progressive mathematization, didactical phenomenology, and self-developed or emergent models. The explanations suggest that he/she correctly evaluated the RME applications conducted in classroom; and coordinated between the approach and its application. It was observed that 6 (18.75%) of prospective teachers incorrectly evaluated the application of RME. 5 of these prospective teachers fall under the category of those who incorrectly comprehended the approach; while 1 of them is in the category of those who correctly comprehended the approach. The prospective teacher, who correctly comprehended the approach but made incorrect statements about its application, made the following statement: “The student is asked real life problems. Afterwards, the student tries to solve this. For example, in the problem about Canyon, various lines were drawn in order to see the river. From what perspective these lines should be coming and the distance between the lines and the river were discussed and tackled. We thought of their ratios and associated them with tangent conception. In conclusion, the teacher makes associations in order to enable the student to solve the problem; and the student solves the problem by bringing them together” (30th PT). In the process of transition from real life to mathematics which is called mathematization, full process of transmitting to mathematical model from original problem situation is referred as modelling (Blum & Niss, 1991). However, in RME, modelling should be conducted by the student (Gravemeijer, 2001; Treffers, 1991). The statements made by the prospective teacher, which expressed that associating should be done by the teacher while solving the problem, suggest that 30 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 he/she is unable (or incorrectly) coordinate between the approach and its in-class application. Analysis In the second question asked to prospective teachers, they were asked to compare RME to other approaches besides discussing about its application. 18 (56.25%) of the prospective teachers correctly analyzed the approach and evaluated its similarities and differences with other approaches. Some of the comments they have made are as follows: “The student reaches at a mathematical conclusion through his/her own knowledge by himself/herself. Since student is in the center of learning and the teacher guides; it is similar to discovery learning. However, in discovery learning, it is not necessary to begin with real life problems. In other words, a student can learn through discovery in a setting, which could enable generalizing a conception, and conduct applications after discovering the conceptions. In fact, in general, this application and constructivist approaches are quite different from traditional education. All of them are focused on the process and student-centered. The students themselves make sense of it. The main difference in RME is the environment is a stimulant and real life problems are what we begin with” (3rd PT). Their explanations suggest that 14 (43.75%) of the prospective teachers are inadequate at analyzing the approach, thus their comparisons of approach to other approaches are sometimes incorrect and sometimes deficient. 6 of these prospective teachers are those, who already incorrectly applied the approach, and other 8 are among those who correctly applied it. One of the students, whose comparisons of other approaches are deficiently, explains it as: “I think there is not a huge difference between them. All of them are student-centered. Some of them create a problem while some of them do the same through an activity instead of a problem. In conclusion, they all end the same” (14th PT). Constructivism, which deals with how knowledge emerges and is based on cognitive psychology, takes the relationships among complex problem solving and cognitive structures and behaviours as basis (Noddings, 1990) and is based on the idea that knowledge is not directly taken from the teacher but structured actively by the learner (Lesh, Doerr, Carmona, & Hjalmarson, 2003; Von Glasersfeld, 1987). RME fundamentally share similarities with various learning and teaching theories of constructivism, which can be considered to be a theory of knowledge. In discovery learning, which is founded by Bruner (1961) and is one of the constructivist approaches, the learner is not informed of target knowledge or the conception, and learning setting is prepared by ensuring proper circumstances (Alfieri, Brooks, Aldrich, & Tenenbaum, 2011). Learning occurs through the assumptions made by organizing patterns and examples, which are based on conceptualizations and generalizations from simple to complex; and conducting researches in an intuitive and systematic Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 31 manner (Jacobsen, Eggen, & Kauchak, 1993) furthermore, in this process teacher's duty is to guide students (Hammer, 1997; Svinicki, 1998). In RME, even though the student is not informed of target knowledge; basic differences in main principles and learning setting play a major role in differentiating approaches form one another. Primarily, basic differences in RME are based on the idea that human knowledge structures the knowledge and mathematical intuitions and procedures are invented and not discovered (Freudenthal, 1973). Thus, even though 'guided reinvention', which is one of the main principles of RME, stresses on guidance in the process; it is still different from the other approaches from that perspective. Moreover, the meaning of guidance in RME refers to the facilitative role of teacher for reinvention while enabling scaffolding instead of making explanations to the students (Hamzah & Bustang, 2014). Essentially, RME and social constructivism, which focuses on the impact of the setting, are concerned with whether student is active; creativity; problem solving; reality of contexts; mathematical reality; and emergence of mathematical objects. However, in RME it is necessary to not only motivate students with everyday life contexts; but also to associate with experimentally real contexts and use them as the starting point for progressive mathematization (Gravemeijer, 2001). Examining the views of the prospective teachers on differences of RME approach from other approaches; the fact that more than half of them mentioned these basic differences suggests that they can analyze at connection-building level in terms of cognitive competency. Reflection Synthesis The prospective teachers were asked to pose a contextual problem (4th question) that they could use in RME, in order elaborately examine the cognitive status of the prospective teachers about RME. Thereby, above the theoretical and practical knowledge they have about the approach; it was aimed to find out how they bring this knowledge together in a different manner and integrate it with associations. Examining the problem they posed, 11 (34.35%) of the prospective teachers produced a contextual problem, in which they could apply the approach; whereas 21 (65.625%) of them did not pose such a problem. Some of the contextual problems are stated below: When these problems are examined, it becomes visible that the students can solve these problems and mathematize new conceptions through the models they created. “Ayşe will meet her friends and go to a movie this weekend. However, she cannot decide what to wear. When she looks at her closet she sees alternatively blouses, skirts, pairs of shoes, and socks with different colors. How many different outfit combinations can she create by doing these pieces in her closet?” Conception: Counting rule-factorial (2nd PT). 32 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 “Uncle Ahmet wants to cover his rectangular garage with square tiles. He measures the size of his garage and goes to buy the tiles from a construction market. However, Uncle Ahmet believes that the less number of tiles, the less it will cost him. For that job, what dimension of tiles Uncle Ahmet should buy”? Conception: The greatest common factor (6th PT). “A medical aid helicopter taking off from the emergency aid base brings aid to those injured in a traffic accident. If the pilot knows the distance from emergency aid base to the hospital, the distance from the scene of accident to the hospital and the angle between them, how can he find the distance between the emergency aid base and the scene of accident?” Conception: Cosine Law (12th PT). The two problems stated below, which were posed on the mathematization of proportion conception by the prospective teachers, are inappropriate for the approach. Although these problems exist in real life, the modeling that will be conducted in the process of solution will not help the mathematization of any conception. Moreover, it is believed that these problems can be an exercise question, which could be applied on taught proportion-ratio conception. “Ezgi's watch loses time 5 minutes every hour. At 10.00 o'clock Ezgi agreed to meet her friend Mehmet at 17.00. What time Ezgi should be there according to her time, in order to meet Mehmet punctually at the time they agreed to meet?” (26th PT). “6 workers worked at a construction and completed the building in 20 days. These workers want to make a planning for their next job. According to this, if 6 workers work together again; in how many days can they finish 5 buildings?” (8th PT). Similarly, even though the problem stated below can exist in real life; it cannot be qualified as a contextual problem. “How many pieces can we cut the cake by 5 moves?” (29th PT). It is believed that the below-mentioned problem cannot be evaluated as a contextual problem, which could serve as a real life problem and help teaching a conception; it rather aims at achieving a mathematical relation by making students generalize through different cases. “Four children play with one hoop each. Later, the children pile hoops over one another and start to examine their junction points. What is the maximum number of junction points for these hoops?” (23th PT). Posing a problem is a significant component for problem solving and good mathematical problems can be posed by good mathematics teachers (Kilpatrick, 1987). The person in the process of posing a problem is actively engaged in challenging situations that involve them in exploring, Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 33 questioning, constructing, and refining mathematical ideas and relationships (English, 2003). In light of this view, it is believed that most of the prospective teachers have carried out the activities mentioned in the process of problem posing. However, a problem should point out the types of realistic thinking, which characterizes out-of-classroom problem solving, in order to be realistic (Verschaffel, Greer, & De Corte, 2000). Also, it should bring out a variety of mathematical interpretations and solution strategies which serve as a basis for progression to a more formal and sophisticated mathematics, and should support students’ mathematisation process (Widjaja, 2013). For example, even though a problem such as “Do the medians of a triangle intersect at a single point” is concrete; it is still far from everyday life problems, thus cannot be considered realistic. Hence, all contextual problems are not realistic. In order to evaluate a problem as realistic, it has to be real or experienced by the person in an interesting manner (Wubbels, Korthagen, & Broekman, 1997). Similarly, if a person wants to pose a problem about a farm, using sentences such as 'imagine a cow in a sphere form' will not make the problem realistic (Greer, 1997). Taking into account of all of these ideas, it is observed that more than half of the problems posed by the prospective teachers are not qualified enough to serve for RME and only 34% of them fit for the purpose. Thus, it can be accepted that only the students in this percentage have cognitively reached the synthesis stage at reflection-construction level. Evaluation In the 4th question asked to the prospective teachers, they were expected to evaluate the application of the problem they posed and interpret it according to the other approaches. The 5 (15.625%) of the prospective teachers, who posed a relevant problem, have carried out an evaluation; whereas the other 27 (84.375%) of them failed to either pose a relevant problem or to carry out an evaluation. The evaluation made by a prospective teacher, who is believed to have posed a relevant problem to RME is as follows: “I aimed to enable the students to achieve the greatest common factor in my problem. The teacher could try to make student comprehend the conception through explaining its meaning. In other words, if it were given through presentation, the student would be provided with prepared knowledge. Furthermore, since the student would not be active; he/she may be bored and mathematics would be far from real life for him/her. I believe that the things a person does or achieves on his/her own is always more valuable and not forgotten. Therefore, for a student to solve a problem, a real problem, and to meet a new conception will be more meaningful for him/her. We could also give this knowledge through discovering numbers and the correlations among factors, without using a problem. This may be effective, too. However, I believe that it will be more permanent for students to reach at knowledge through real life events, discussion and creating their own models” (6th PT). 34 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 According to Hiebert and Carpenter (1992), understanding is defined as making connections between ideas, facts or procedures, and occurs through recognizing relationships between pieces of knowledge. Consolidated knowledge enables the use of this knowledge in various cases in a proper and confident manner (Wubbels, Korthagen, & Broekman; 1997). Also, if knowledge is consolidated; the evaluations about this knowledge will be conducted in a meaningful manner. In that sense, it can be concluded that the evaluations of prospective teachers through the problems they posed are insufficient. CONCLUSION Enabling students to learn mathematics in a meaningful manner and to acquire various targeted skills during the process can be only made possible by placing education of mathematics teachers in the center. Therefore, it is of vital importance for prospective teachers to be able to prepare appropriate learning environments and to correctly plan the roadmaps for that purpose (Brousseau, 1987; Richards, 1991). In order to achieve that, the prospective teachers must have enough knowledge on the key principles and fundamental philosophy of the approach that they plan on using while preparing such environments, moreover, they must also be able to implement and interpret instructional activities (Gravemeijer & Cobb, 2006). In this study, where prospective teachers’ cognitive structures about the approaches have been examined, RME has been the focused approach and prospective teachers’ cognitive structures about this approach have been evaluated. The study suggests that the majority of the prospective teachers have theoretical knowledge about RME and can generally interpret that knowledge, however, their ability to associate that knowledge with other approaches falls by half. Moreover, the prospective teachers’ skills fell even shorter when they were asked to pose a contextual problem suitable with RME, which was asked of them in order to see their high cognitive competencies. Even though it is necessary to place mathematical connections in the relevant social situations for achieving meaningful learning in mathematics; creating authentic activities is naturally quite difficult (Ainley, Pratt, & Hansen, 2006) and also designing lesson materials, especially finding real life examples that match with the mathematics concepts to be taught is difficult (Zulkadri, 2002). Also, preparing a learning environment that allows for reinvention process, is quite complicated (Gravemeijer, 2008). Furthermore, some of the reasons why these problems emerged may be due to the fact that they do not have adequate contextual knowledge about the nature of some of the conceptions they were supposed to learn; or they could not detach from the constructions in the learning settings, in which they were raised. Also, as Widjaja (2008) mentioned, their prior knowledge about the concept and the nature of their knowledge may cause this result. For that reason, prospective teachers should be trained in a way that reflects what is expected from their teachings (Gravemeijer, 2008). Since this study aims to reveal the cognitive competency of prospective teachers on RME; it is believed that the obtained results suggest ideas on the construction about RME only in the minds of Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 35 prospective teachers. Hence, reflections of prospective teachers on RME can be conducted through a phenomenological study so as to obtain more elaborated knowledge. Moreover, in the school experience and school practice stages; the application processes of prospective teachers on RME can be examined and in the light of the retrieved results, necessary adaptations can be conducted for real classroom situations. ACKNOWLEDGMENTS I would like to thank the prospective teachers for their voluntary participation. REFERENCES Ainley, J., Pratt, D., & Hansen, A. (2006). Connecting engagement and focus in pedagogic task design. British Educational Research Journal, 32(1), 23-38. https://doi.org/10.1080/01411920500401971. Alfieri, L., Brooks, P.J., Aldrich, N.J., & Tenenbaum, H.R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1-18. https://doi.org/10.1037/a0021017. Altun, M. (2011). Eğitim fakülteleri ve matematik öğretmenleri için liselerde matematik öğretimi (Mathematics Education in High Schools). Bursa: Alfa Akademi. Anderson, L.W., & Krathwohl, D.R. (2001). (Eds.) Taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Needham Heights, MA: Allyn & Bacon. Anderson, J., White, P., & Wong, M. (2012). Mathematics curriculum in the schooling years. In B. Perry, T. Lowrie, T. Logan, A. MacDonald, & J. Greenlees (Eds.), Research in mathematics education in Australasia 2008–2011 (pp. 219–244). Rotterdam: Sense Publishers. Aubrey, K., & Riley, A. (2016). Understanding and using educational theories. London: Sage. Ausubel, D.P. (1963). The psychology of meaningful verbal learning. New York: Gruene and Stratton. Ayer, A.J. (1936). Language, truth and logic. Oxford: Oxford University Press. Barell, J. (2010). Problem-based learning: The foundation of 21st century skills. In J. Bellance & R. Brandt R (Eds.), 21 st century skills (pp. 174-199). Bloomington: Solution Tree Press. Barnes, H. (2004). Realistic mathematics education: Eliciting alternative mathematical conceptions of learners. African Journal of Research in Mathematics, Science and Technology Education, 8(1), 53-64. https://doi.org/10.1080/10288457.2004.10740560. Barrett, T.C. (1976). Taxonomy of reading comprehension. In R. Smith & T. C. Barrett (Eds.). Teaching reading in the middle grades. Reading, MA: Addison-Wesley Publishing Co. Barrows, H.S. (1986). A taxonomy of problem‐based learning methods. Medical Education, 20(6), 481-486. https://doi.org/10.1111/j.1365-2923.1986.tb01386.x. Battista, M.T. (1994). Teacher beliefs and the reform movement in mathematics education. The Phi Delta Kappan, 75(6), 462-470. Baumert, J., & Kunter, M. (2013). The effect of content knowledge and pedagogical content knowledge on instructional quality and student achievement. In Cognitive activation in the 36 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 mathematics classroom and professional competence of teachers (pp. 175-205). Boston: Springer. Berg, B.L. (2001). Qualitative research methods for the social sciences (4th ed.). Boston: Allyn & Bacon. Beswick, K. (2011). Putting context in context: An examination of the evidence for the benefits of ‘contextualised’tasks. International Journal of Science and Mathematics Education, 9(2), 367390. http://dx.doi.org/10.1007/s10763-011-9323-y. Beswick, K., & Goos, M. (2012). Measuring pre-service primary teachers' knowledge for teaching mathematics. Mathematics Teacher Education and Development, 14(2), 70-90. Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay. Bloom, B.S. (1994). Reflections on the development and use of the taxonomy. In L. W. Anderson & L. A. Sosniak (Eds.). Bloom's taxonomy: A forty-year retrospective (1-61). Chicago: University of Chicago Press. Blum, W., & Niss, M. (1991). Applied mathematical problem solving, modelling, applications, and links to other subjects-state: trends and issues in mathematics instruction. Educational Studies in Mathematics, 22(1), 37-68. https://doi.org/10.1007/BF00302716. Borko, H., & Putnam, R. (1996). Learning to teach. In D. C. Berliner, R. C. Calfee (Eds.). Handbook of educational psychology (673-708). New York: Simon & Schuster Macmillan. Borasi, R., & Fonzi, J. (2002). Professional development that supports school mathematics reform. Foundations series of monographs for professionals in science, mathematics and technology education. Arlington, VA: National Science Foundation. Bray, A., & Tangney, B. (2016). Enhancing student engagement through the affordances of mobile technology: a 21st century learning perspective on Realistic Mathematics Education. Mathematics Education Research Journal, 28(1), 173-197. https://doi.org/10.1007/s13394-015-0158-7. Brousseau, G. (1987). Les differents roles du maitre. Colloque des P.E.N. Angers. In S. A. Martin (Ed.). Reconstructing mathematics pedagogy from a constructivist perspective. Washington, DC: National Science Foundation. Bruner, J.S. (1961). The act of discovery. Harvard Educational Review, 31, 21-32. Cai, J., & Howson, G. (2013). Toward an international mathematics curriculum. In M. A. K. Clements, A. J. Bishop, C. Keitel, J. Kilpatrick, & F. K. S. Leung (Eds.), Third international handbook of mathematics education (pp. 949–974). New York: Springer. Creswell, J.W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousands Oaks, CA: Sage. Davis, R.B. (1990). Discovery Learning and Constructivism. Journal for Research in Mathematics Education: Monograph (4): Constructivist Views on the Teaching and Learning of Mathematics, (pp. 93-106+195-21). Association Drive, Reston, VA 22091. Dawkins, P.C. (2015). Explication as a lens for the formalization of mathematical theory through guided reinvention. The Journal of Mathematical Behavior, 37, 63-82. Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 37 De Corte, E. (1995). Fostering cognitive growth: A perspective from research on mathematics learning and instruction. Educational Psychologist, 30(1), 37-46. https://doi.org/10.1207/s15326985ep3001_4. De Lange, J. (1987). Mathematics, insight, and meaning. The Netherlands, Utrecht: OW&OC. De Lange, J. (1996). Using and applying mathematics in education. In A. J. Bishop et al. (Eds.). International handbook of mathematics education (pp. 49-97). The Netherlands: Kluwer Academic Publishers. Doorman, M., Drijvers, P., Dekker, T., van den Heuvel-Panhuizen, M., de Lange, J., & Wijers, M. (2007). Problem solving as a challenge for mathematics education in the Netherlands. ZDM, 39(5-6), 405-418. https://doi.org/10.1007/s11858-007-0043-2. Dreyfus, T., & Tsamir, P. (2004). Ben's consolidation of knowledge structures about infinite sets. The Journal of Mathematical Behavior, 23(3), 271-300. https://doi.org/10.1016/j.jmathb.2004.06.002. Driscoll, M.P. (2005). Psychology of learning for instruction. (3th Ed.) Boston, MA: Pearson. English, L.D. (1995). Cognitive psychology and mathematics education. In L. D. English, G. S. Halford (Eds.). Mathematics education: Models and processes (pp. 1-20). Lawrence Erlbaum Associates, Mahwah: NJ. English, L.D. (2003). Problem posing in the elementary curriculum. In F. Lester, R. Charles (Eds.). Teaching mathematics through problem solving. Reston, Virginia: National Council of Teachers of Mathematics. Ertmer, P.A., & Newby, T.J. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50-72. https://doi.org/10.1111/j.1937-8327.1993.tb00605.x Fauzan, A. (2002). Applying Realistic Mathematics Education (RME) in teaching geometry in Indonesian primary schools. Unpublished Doctoral Dissertation. Enschede: University of Twente. Feijs, E. (2005). Constructing a learning environment that promotes reinvention. In R. Nemirovsky, A. S. Rosebery, J. Solomon, & B. Warren (Eds.). Everyday matters in science and mathematics: Studies of complex classroom events. Mahwah, NJ: Lawrence Erlbaum. Feiman-Nemser, S. (2001). From preparation to practice: Designing a continuum to strengthen and sustain teaching. Teachers College Record, 103(6), 1013-1055. Freudenthal, H. (1973). Mathematics as an educational task. Dordrecht: Reidel. Freudenthal, H. (1991). Revisiting mathematics education. Dordrecht: Kluwer Academic. Granello, D. H. (1995). Encouraging the cognitive development of supervisees: Using Bloom's taxonomy in supervision. Counselor Education and Supervision, 40(1), 31-46. https://doi.org/10.1002/j.1556-6978.2000.tb01797.x. Granello, D. H. (2001). Promoting cognitive complexity in graduate written work: Using Bloom’s taxonomy as a pedagogical tool to improve literature reviews. Counselor Education and Supervision, 40, 292-307. https://doi.org/10.1002/j.1556-6978.2001.tb01261.x. Gravemeijer, K. (1999). How emergent models may foster the constitution of formal mathematics. Mathematical Thinking and Learning, 1(2), 155-177. https://doi.org/10.1207/s15327833mtl0102_4. 38 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 Gravemeijer, K. (1994). Developing realistic mathematics education. Utrecht: Utrecht, The Netherlands: CD-β Press. Gravemeijer, K.P.E. (2001). Developmental research: Fostering a dialectic relation between theory and practice. In J. Anghileri (Ed.). Principles and practice in arithmetic teaching (147-161). London: Open University Press. Gravemeijer, K.P.E., & Cobb, P. (2006). Design research from a learning design perspective. In J.Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 45–85). London: Routledge, Taylor Francis Group. Gravemeijer, K. (2008). RME theory and mathematics teacher education. International handbook of mathematics teacher education, 1, 283-302. Greer, B. (1997). Modelling reality in mathematics classrooms: The case of word problems. Learning and Instruction, 7(4), 293-307. https://doi.org/10.1016/S0959-4752(97)00006-6. Hadi, S. (2002). Effective teacher professional development for the implementation of realistic mathematics education in Indonesia. Unpublished Doctoral Dissertation. University of Twente. Hammer, D. (1997). Discovery learning and discovery teaching. Cognition and Instruction, 15(4), 485-529. https://doi.org/10.1207/s1532690xci1504_2. Hamzah, U., & Bustang, B. (2014). Constructivism versus cognitive load theory: in search for an effective mathematics teaching. In Proceeding of International Conference on Research, Implementation and Education of Mathematics and Sciences, May 18-20, 2014. Yogyakarta State University. Herman, P., & Gomez, L.M. (2009). Taking guided learning theory to school. In S. Tobias, D. M. Duffy (Eds.). Constructivist theory applied to instruction: Success or failure? (pp. 62-81). New York, NY: Taylor & Francis. Hiebert, J., & Carpenter, T.P. (1992). Learning and teaching with understanding. In J. Hiebert, T. P. Carpenter, D. A. Grouws (Eds.). Handbook of research on mathematics teaching and learning: A project of the National Council of Teachers of Mathematics (pp. 65-97). New York: Macmillan. Jacobsen, D., Eggen, P., & Kauchak, D. (1993). Methods for teaching: A skills approach (4th ed.). Columbus, OH: Merrill. Jonassen, D.H. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational technology research and development, 39(3), 5-14. https://doi.org/10.1007/BF02296434. Jonassen, D.H. (1991a). Evaluating constructivistic learning. Educational Technology, 31(9), 28-33. Kilpatrick, J. (1987). Problem formulating: Where do good problems come from? In A. H. Schoenfeld (Ed.). Cognitive science and mathematics education (123-148). Hillsdale. NJ: Erlbaum. Klein, A.S. (1998). Flexibilization of mental arithmetic strategies on a different knowledge base. Utrecht: Freudenthal Institute. Kraiger, K., Ford, J.K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Pschology, 78(2), 311-328. Krathwohl, D.R., Bloom, B.S., & Bertram, B.M. (1973). Taxonomy of educational objectives, the classification of educational goals. Handbook II: affective domain. New York: David McKay. Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 39 Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An Overview. Theory into Practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2. Kwon, O.N. (2002). Conceptualizing the Realistic Mathematics Education approach in the teaching and learning of ordinary differential equations. In Proceedings of the International Conference on the Teaching of Mathematics Hersonissos, July 1-6. Crete, Greece. Lesh, R., Doerr, H.M., Carmona, G., & Hjalmarson, M. (2003). Beyond constructivism. Mathematical Thinking and Learning, 5(2&3), 211–233. http://dx.doi.org/10.1080/10986065.2003.9680000. Lianghuo, F. (2014). Investigating the pedagogy of mathematics: How do teachers develop their knowledge?. Imperial College Press, London. Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry (Vol. 75). California: Sage. Makonye, J. P. (2014). Teaching functions using a realistic mathematics education approach: A theoretical perspective. International Journal of Educational Sciences, 7(3), 653-662. https://doi.org/10.1080/09751122.2014.11890228. National Research Council. (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, & B. Findell (Eds.). Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. Noddings, N. (1990). Constructivism in mathematics teaching. In R. B. Davis, C. A. Maher, N. Noddings (Eds). Constructivist views on the learning and teaching of mathematics (7-18). Reston, VA: National Council of Teachers of Mathematics. (J for Res in Math Edu. Monograph Number 4). OECD. (2003). The PISA 2003 assessment framework. mathematics, reading, science and problem solving knowledge and skills. Paris: OECD. OECD. (2009). PISA 2006 Technical report. Paris: OECD. Patton, M.Q. (1990). Qualitative evaluation and research methods. (2nd ed.) Newbury Park, CA: Sage. Pearson, P.D., & Johnson, D.D. (1978). Teaching reading comprehension. New York: Holt, Rinehart, & Winston. Piaget, J. (1973). To understand is to invent: The future of education. New York: Grossman. Putnam, R.T., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Research, 29(1), 4-15. https://doi.org/10.3102%2F0013189X029001004. Rasmussen, C.L., & King, K.D. (2000). Locating starting points in differential equations: A realistic mathematics education approach. International Journal of Mathematical Education in Science and Technology, 31(2), 161-172. https://doi.org/10.1080/002073900287219. Richards, J. (1991). Mathematical Discussion. In E. von Glasersfeld (Ed.). Radical constructivism in mathematical education (pp. 13-52). Dortrecht: Kluwer. Sadler, T.D., & Zeidler, D.L. (2009). Scientific literacy, PISA, and socioscientific discourse: Assessment for progressive aims of science education. Journal of Research in Science Teaching, 46(8), 909-921. https://doi.org/10.1002/tea.20327. 40 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 Selter, C. (1997). Instructional design for teacher education. In M. Beishuizen, K.P.E. Gravemeijer, & E.C.D.M. van Lieshout (Eds.), The role of contexts and models in the development of mathematics strategies and procedure (pp. 55-77). Utrecht:CD-B Press/Freudenthal Institute. Selter, C. (2001). Understanding-the underlying goal of teacher education. In M. Van den HeuvelPanhuizen (Ed.), Proceeding of 25th conference international group of psychology of mathematics education, 12-17 July, Utrecht, the Netherlands. Sembiring, R.K, Hadi, S., & Dolk, M. (2008). Reforming mathematics learning in Indonesian classrooms through RME. ZDM, 40, 927-939. https://doi.org/10.1007/s11858-008-0125-9. Simon, M.A. (1995). Reconstructing mathematics pedagogy from a constructivist perspective. Journal for Research in Mathematical Education, 26(2), 114-145. https://doi.org/10.2307/749205. Simons, R.J., Van der Linden, J., & Duffy, T. (Eds.) (2000). New learning. Dordrecht, The Netherlands: Kluwer Academic Publishers. Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard educational review, 57(1), 1-23. Sowder, J.T. (2007). The mathematics education and development of teachers. In F. K. Lester Jr. (Ed.), Second handbook of research on mathematics teaching and learning (pp. 157-223). Charlotte, NC: Information Age. Sriraman, B., & English, L. (2010). Surveying theories and philosophies of mathematics education. In B. Sriraman & L. English (Eds). Theory of mathematics education. Advances in mathematics education (pp. 7-32). Berlin Heidelberg: Springer-Verlag. Stacey, K. (2005). The place of problem solving in contemporary mathematics curriculum documents. Journal of Mathematical Behavior, 24, 341–350. https://doi.org/10.1016/j.jmathb.2005.09.004. Streefland, L. (1991). Fraction in Realistic Mathematics Education, A paradigm of developmental research. Dordrecht: Kluwer Academic Publisher. Svinicki, M.D. (1998). Theoretical Foundation for Discovery Learning. Advances in Psychological Education, 20(1), S4-S7. https://doi.org/10.1152/advances.1998.275.6.S4. Szydlik, J.E., Szydlik, S.D., & Benson, S.R. (2003). Exploring changes in pre-service elementary teachers’ mathematical beliefs. Journal of Mathematics Teacher Education, 6(3), 253-279. https://doi.org/10.1023/A:1025155328511 Treffers, A., & Goffree, F. (1985). Rational analysis of realistic mathematics education - The Wiskobas Program. In L. Streefland (Ed). Proceedings of the 9th International Conference for the Psychology of Mathematics Education, July 22-29; Noordwijkerhout. Utrecht, The Netherlands. Treffers, A. (1991). Realistic mathematics education in the Netherlands 1980 - 1990. In L. Streefland (Ed.). Realistic Mathematics Education in primary schools (pp. 11–20). Utrecht: Freudenthal Institute. Treffers, A. (1987). Three dimensions. A model of goal and theory description in mathematics education. Dordrecht: Reidel. Tsamir, P. (2008). Using Theories as Tools in Mathematics Teacher Education, In D. Tirosh & T. Wood (Eds.). Tools and Processes in Mathematics Teacher Education (pp. 221-232). Rotterdam: Sense Publishers. Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 41 Van den Heuvel-Panhuizen, M. (1996). Assessment and Realistic Mathematics Education. Utrecht: Freudenthal Institute. Van den Heuvel-Panhuzen, M. (2003). The didactical use of models in realistic mathematics education: An example from a longitudinal trajectory on percentage. Educational Studies in Mathematics, 54, 9-35. https://doi.org/10.1023/B:EDUC.0000005212.03219.dc. Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. Swets & Zeitlinger, Lisse. Von Glasersfeld, E. (1987). Learning as a constructive activity, In C. Janvier (Ed.). Problems of representation in the teaching and learning of mathematics (pp. 3-18). Hillsdale, NJ: Lawrence Erlbaum Associate. Wang P.L. (1990). Assessment of Cognitive Competency. In D. E. Tupper and K. D. Cicerone (Eds.) The Neuropsychology of Everyday Life: Assessment and Basic Competencies. Foundations of Neuropsychology (pp. 219-228). Springer, Boston, MA. Webb, N. (1997). Criteria for alignment of expectations and assessments on mathematics and science education. Council of Chief State School Officers and National Institute for Science Education Research Monograph No. 6. Washington, DC: CCSSO. Webb, N. (1999). Alignment of science and mathematics standards and assessments in four states. Council of Chief State School Officers and National Institute for Science Education Research Monograph No.18. Washington, DC: CCSSO. Widjaja, W. (2008). Local instruction theory on decimals: The case of Indonesian pre-service teachers. Doctoral dissertation. Melbourne: The University of Melbourne. Widjaja, W. (2013). The use of contextual problems to support mathematical learning. Journal on Mathematics Education, 4(2), 157-168. http://dx.doi.org/10.22342/jme.4.2.413.151-159. Wu, H. (1999). Basic skills versus conceptual understanding. American Educator, 23(3), 14-19. Wubbels, T., Korthagen, F., & Broekman, H. (1997). Preparing teachers for realistic mathematics education. Educational Studies in Mathematics, 32, 1–28. https://doi.org/10.1023/A:100290052. Yackel, E., & Cobb, P. (1996). Sociomath norms, argumentation, and autonomy in mathematics. Journal for Research in Mathematical Education, 27, 75-190. https://www.jstor.org/stable/749877 Yackel, E., Gravemeijer, K., & Sfard, A. (2011). A journey in mathematics education research. Dordrecht: Springer. Yıldırım, A., & Şimşek, H. (2005). Sosyal bilimlerde nitel araştırma yöntemleri (Qualitative research methods for social sciences). Ankara: Seçkin Yayıncılık. Zulkardi. (2002). Developing a learning environment on realistic mathematics education for Indonesian student teachers. Doctoral Dissertation. Enschede: University of Twente. 42 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 APPENDIX Problem 1 When the sea snake in the picture becomes 1 month old, a black ring emerges around its body. Each month, a yellow ring emerges in the middle of the black ring; and thereby, two black and one yellow rings appear. In the following months, this continues in the same way. In other words, each black ring is cut in the middle with a yellow ring. How many rings does a 12 month old sea snake have? How can we find the age of a sea snake according to its number of rings; or the number of black and yellow rings of a sea snake of a certain age? (Altun, 2011). Comment: It was aimed to reach geometrical sequence conception through this problem. Problem 2 Ship P is going in a straight line toward point B on the shore, at a constant speed. Ship Q is going in the direction of A at double the speed of ship P. How close do the two ships get? (Wubbels, Korthagen, & Broekman, 1997). Comment: It was aimed to reach vector conception through this problem. Problem 3 The figure below shows two rice fields separated by a road. Both rice fields are planted with the same rice and they are given the same fertilizer. The dots on the figure represent rice clusters. Which rice field produces more rice? (Fauzan, 2002). Comment: It was aimed to reach reallotment-congruent area conception through this problem. Yilmaz, Prospective mathematics teachers’ cognitive competencies on realistic mathematics education 43 Problem 4 This is a photograph of a hiker on the rim of the Grand Canyon looking down trying to see the Colorado River at the bottom of the canyon. Can the hiker see the river below? From which points and perspective should the hiker look in order to see the river? (Feijs, 2005). Comment: It was aimed to discuss tangent conception through this problem. 44 Journal on Mathematics Education, Volume 11, No. 1, January 2020, pp. 17-44 International Journal on Emerging Mathematics Education (IJEME) Vol. 1, No. 1, March 2017, pp. 11-24 P-ISSN: 2549-4996, E-ISSN: 2548-5806, DOI: http://dx.doi.org/10.12928/ijeme.v1i1.5695 Teachers’ Real and Perceived of ICTs Supported-Situation for Mathematics Teaching and Learning 1Maman Fathurrohman, 2Anne L. Porter, 2Annette L. Worthy 1Universitas Sultan Ageng Tirtayasa, Jl. Raya Jakarta Km. 4, Panancangan, Serang, Banten 42124 2University of Wollongong, Northfields Ave, Wollongong NSW 2522, Australia Email: [email protected] Abstrak Tujuan dari penelitian ini adalah untuk mendapatkan informasi yang berkaitan dengan Infrastuktur, Fasilitas, dan Sumber Daya Teknologi Informasi dan Komunikasi (TIK) di Kecamatan Bojonegara, Provinsi Banten, Indonesia. Paper ini menekankan publikasi hasil tersebut karena informasi itu diperlukan untuk penerapan pembelajaran matematika berbasis teknologi. Metode yang digunakan adalah survai. Instrumen yang digunakan adalah kuesioner, pedoman wawancara tidak terstruktur, dan handycam. Selama survei, total 220 paket kuesioner dibagikan kepada guru, tetapi hanya 119 (tingkat respon 54,1%) yang diisi dan dikembalikan. Sebanyak 12 guru telah diwawancarai dengan lima dari wawancara tersebut direkam dalam video. Beberapa kepala sekolah menyambut baik dan mengizinkan peneliti untuk mengunjungi sekolah mereka dan membuat dokumentasi terkait dengan infrastruktur, fasilitas, dan sumber daya TIK di sekolah mereka, sementara yang lain tidak mengizinkan peneliti untuk melakukannya. Berdasarkan survei, berbagai fakta penting telah ditemukan. Disarankan bahwa Teacher-Centered Learning with Technology adalah metode yang paling tepat untuk diterapkan. Kata Kunci: survai, pembelajaran berbasis teknologi, TIK, infrastruktur, fasilitas, sumber daya Abstract Purpose of this research was to gain information related to the real and perceived ICT Infrastructures, Facilities, and Resources in Bojonegara Sub District, Indonesia. This article emphasize the publication of this information because it is needed for implementation of technology-based mathematics teaching and learning. The method used was survey. Instruments of survey were questionnaires, unstructured interview guideline, and handycam. During the survey, total of 220 questionnaire packages were distributed to teachers, however only 119 (response rate 54.1%) of them were filled and returned. A total of 12 teachers were interviewed, with five of these interviews were video recorded. Several head masters welcomed and allowed researcher to visit their schools and make documentation of ICT Infrastructures, facilities, and resources, while the others did not allow the researcher to do that. Based on survey, many important findings have been discovered. It is suggested that the Teachers-Centered Learning with Technology is the most appropriate method of technology-based learning to be implemented. Keywords: survey, technology-based learning, ICT, infrastructure, facilities, resources How to Cite: Fathurrohman, M., Porter, A.L., & Worthy, A.L. (2017). Teachers’ real and perceived of ICTs supported-situation for mathematics teaching and learning. International Journal on Emerging Mathematics Education, 1(1), 11-24. http://dx.doi.org/10.12928/ijeme.v1i1.5695. INTRODUCTION Technology is one aspect in human civilization that spreads quickly from one nation to other nations. The spreads of technology, especially information and communication technology, has influence and change human life. In the field of education, technology has been adopted as can be seen in the daily academic activities, not only in universities but also in Received January 24, 2017; Revised February 16, 2017; Accepted February 23, 2017 12 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 secondary and primary schools. Teaching and learning with technology have been happens in many places because of its benefit on provides advantage in making the process of teaching and learning more effective and efficient. The implementation of technology-based mathematics teaching and learning requires supporting infrastructures, facilities, and resources. For that reason, the conditions of these components, called the ICTs Supported-Situation was needed for implementation of technologybased learning, have to be known. Several initiatives have been conducted to discover ICTs Supported-Situation for Teaching and Learning. The initiative were surveys of, or related to, ICT for education with reports generating recommendations, including recommendation regarding ICT for education. Examples include, The Survey of ICT and Education and country focused surveys in ten countries, Argentina, Brazil, the Philippines, Kenya, India, Morocco, Peru, Senegal, Ukraine, and Vietnam, that also provide data and information about recent ICT conditions in these developing countries, and surveys of India and seven other developing countries in South Asia, Afghanistan, Bangladesh, Bhutan, Maldives, Nepal, Pakistan, and Sri Lanka by Information and Development Program or InfoDev (2007; 2009a;2009b; 2010). The published reports of surveys or in part of survey by the World Economic Forum (2016) and International Development Research Centre (2010) also comprehensively addressed current usage of ICT. Some sections of reports briefly discussed regional and global trends of ICT in education and selected regional ICT initiatives and projects in education. In addition to the surveys several studies related to ICTs Supported-Situation and education. However the surveys are too general to be practically used or refereed for implementation of technology-based mathematics teaching and learning. There is a need to conduct a survey in accessible area for this purpose. Term infrastructure, facility, and resource are defined as following: Infrastructure is defined as the basic physical and organizational structures and facilities needed for the operation of a society or enterprise. Facility is defined as a place, amenity, or piece of equipment provided for a particular purpose. While Resource is defined as a stock or supply of money, materials, staff, and other assets that can be drawn on by a person or organization in order to function effectively. The implementation of technology is relevant to Fathurrohman (2012) suggestion for Addressing the need of mathematics teachers in developing countries. Based on above definitions, researchers defined the objects of study which were needed to be known. For infrastructures were the conditions of ICT infrastructures at the area, the schools, and teachers’ home. For facilities were the computer lab, computer, printer, scanner, internet connection, and handphone at schools and teachers home. For resources were digital materials, CD, DVD, and software related to it. One important thing that needs to be considered was the perspective of teachers related to above components. In other word, not only the real condition, but also the perceived condition, based on teachers’ perspective about these components was needed to be taken into account. According to Oxford Dictionaries (2011), the Real is defined as actually existing as a thing or occurring in fact; not imagined or supposed. While Perceive is defiend as become aware or conscious of (something); come to realize or understand. REAL PERCEIVED Infrastructures Infrastructures Facilities Facilities Resources Resources Figure 1. Objects of Survey IJEME, Vol. 1, No. 1, March 2017, 11-24 IJEME ISSN: 2549-4996 13 The knowledge related to these components would useful to select a suitable technology-based learning method. According to seminal work by Ross (1995: 24) there are five technology-based learning methods available. These methods are based on the concept of learner control. Table 1. Matrix of technology-based learning methods Learning Method Center of Control World of Work Strategies Example Teacher-centered Teachers: The Training sessions, Multimedia learning with teachers directs specific skill presentation, technology the pace and development videotape, distance sequence instruction Integrated learning Machine: A Teaching machines Distributed ILS, system computer network lab-centered ILS and its software direct the learning Electronic Teams or partners: Developmental Local area collaboration The teams teams, joint networks, wide are learning negotiates, goals, research efforts, networks, pacing, and learning teams cooperative sequence of ventures learning Hyperlearning Learner: The Research, market Hypertext learner is in charge analysis, development, of pace and engineering design hypermedia sequence of development, learning multimedia development, network searching Electronic learning Machine and Flight simulators, Virtuality simulations learner: Learning disaster control electronic is in joint control simulations, war simulation games The table revealed five technology-based learning methods. The table also provides what is the suitable condition to implement related method. The implementation of technology-based learning experiences requires information of current condition of real and perceived ICT infrastructures, facilities, and resources in developing countries. This article emphasize publication of this information because it is needed for implementation of technology-based mathematics teaching and learning. In order to get this information, a survey. RESEARCH METHOD Method of this research is survey. Survey is defined as examine and record the area and features to construct a map, plan, or description (Oxford, 2001). In this research, purpose of this survey is to examine and record the real and perceived infrastructures, facilities, and resources to construct a plan for implementation of technology-based mathematics teaching and learning. According to De Vaus (2002) there are five ethical responsibilities towards survey participants: 1) voluntary participation, 2) informed consent, 3) no harm, 4) confidentiality anonymity, and 5) privacy. The responsibilities already considered for this activities. Teachers’ Real and Perceived of ICTs Supported-Situation … Fathurrohman, Porter, & Worthy 14 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Population and Sample One accessible area and of interest to the researchers was Bojonegara Sub Disctrict, Banten Province, Indonesia. Banten Province is located in Java Island, Indonesia. Respondents of of this research are registered teachers, defined as teachers who have NUPTK (Nomor Urut Pendidik dan Tenaga Kependidikan) at the start of survey. Following is the data related to respondents Table 2. Data of registered teachers in Bojonegara Sub District, Indonesia Number of Number of No Level Percentage Schools teachers 1 Elementary School 22 313 53.50 2 Junior High School 9 194 33.16 3 Senior High School 3 78 13.33 Total 34 585 100.00 To facilitate comparisons and to have a sufficiently large sample the researchers took a higher proportion from the smaller cohorts. Researchers also consider time duration and an access to schools in choosing the number of schools to sample. Following is an estimation of the number each school type. 1. Around 1/7 of number of elementary schools teachers (1/7 x 313 = 45 teachers) 2. Around 1/5 of number of junior secondary schools teachers (1/5 x 194 = 39 teachers) 3. Around 1/2 of number of senior elementary schools teachers (1/2 x 78 = 39 teachers) The teachers were determined based on a simple random sampling scheme within each type. Data and Instruments The collected data including the data related to the supporting infrastructures, facilities, and resources for implementation of technology-based learning. Moreover, because the technology-based learning also related to learning resources and learning design, the data related to these components in teaching and learning experiences also need to be collected. Learning defined as the acquisition of knowledge or skills through study, experience, or being taught, while Design isdefined as purposes or planning that exist behind an action, fact, or object. Started from this, the following definitions are proposed. Resource is defined as a source of help or information, learning is defined as the acquisition of knowledge or skills through study, experience, or being taught, while Mathematical defined as relating to mathematics. Based on above definitions, Mathematical Learning Resources are sources of information that can be used to acquisition of knowledge or skills related to mathematics through study, experience, or being taught. In this article, Mathematical Learning Resources defined as sources of information, represented in a variety of media and format that can be used to assist student learning through study, experience, or being taught, as defined in national curricula, to acquisition of knowledge or skills related to mathematics Learning Designs are planning that exist behind actions or facts of the process acquisition of knowledge or skill through study or experience. Term learning design is IJEME, Vol. 1, No. 1, March 2017, 11-24 IJEME ISSN: 2549-4996 15 variously defined by different authors as a process of, and for, designing learning experiences. The purposes of learning design are to documents and describe learning activity that other teachers can understand it and use it (in some way) in their own context. Specifically, Learning Design defined can be considered in two ways: (1) as a process designing learning experiences and (2) as a product, that is, the outcome or artifact of the design process (Agostinho, 2009, p. 4). To gather data three types of instruments were used. 1. Questionnaire The questionnaire consists of 27 main questions. Several main questions have one or two child questions for further deep information or clarification. The topic of questions is based on focus of survey as described in Table below 2. Unstructured Guideline for Interview Unstructured guideline for interview constructed to follow the flow of discussion with teachers. Interview were conducted by following the flow of discussion. Sometime researcher found a new interesting topic while conducting discussion with teachers. 3. Handycam Handycam used to take pictures and videos related infrastructures, facilities, and resources available in the area. For each teacher, gender, working experience, academic degree & field of study, and training in ICT was assessed in relation to ICT Condition at school & home, and the learning designs & resources they had access to or experiences. Validity of data gathered is ensured through triangulation. Three sources of data based on different instruments for one same of object are used in triangulation. RESULTS AND DISCUSSION Bojonegara is a Sub District of Banten Province, located in a coastal region of Java Island, Indonesia (refer Figure 3). The area, covers 30.30 km2. Figure 2. Location of Bojonegara Sub District, Banten Province, Indonesia Geographically, rural areas dominate Bojonegara Sub District. However ICT technologies have penetrated the area, bring modernity to the people and the coutryside. As illustrated by photos there are many rice paddies, along with clusters of people, houses and buildings, such as government buildings and schools. Teachers’ Real and Perceived of ICTs Supported-Situation … Fathurrohman, Porter, & Worthy 16 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Figure 3. Survey Photos of Bojonegara Sub District, Banten Province, Indonesia Participants The localised study was conducted October to February years ago, with several weeks of break around the end of December and early of January. Ten headmasters welcomed and allowed the researcher to visit and document ICT infrastructure and facilities in their schools. Only two headmasters did not allow the researcher access to the school or teachers. A total of 220 questionnaire packages were distributed to teachers in elementary, junior and senior secondary school. Over the three school levels a total number of 119 teachers completed and returned the questionnaires, giving a response rate of 54 per cent. The numbers of teachers as respondents is not exactly same as sought when the sample composition was calculated, with less responding from the elementary school and senior secondary school level, and more responding from junior secondary school level as presented in the Table 3. The variation from that intended was reasonable in terms of providing groups near 30 teachers and the responses considered viable numbers in terms of representing school ICT conditions. Table 3. Comparison of expected number and the real number of participants Expected to School level Participated Difference Participate Elementary School 45 41 -4 Junior Secondary School 39 51 + 12 Senior Secondary School 39 27 - 12 Total 123 119 -4 The teachers were considered to be participants if they either fully or partially completed the questionnaire then returned it to the researcher. In addition, a total of 12 teachers (some of them completed questionnaire and some did not) agreed to be interviewed, with five of these interviews video recorded. At each school level the respondents are predominantly female and this is in accord with a greater proportion of teachers being female. Approximately 65 per cent of respondents were female. IJEME, Vol. 1, No. 1, March 2017, 11-24 IJEME ISSN: 2549-4996 17 The teachers’ years of teaching/working experiences is presented in Table 5. Notably, approximately 80 per cent of the respondents had more than three years experience. While 43 per cent have 4 to 10 years working experiences. Indeed only 14.6 per cent of elementary teachers are in their first three years of teaching, although this increases to 20.6 per cent junior secondary and 29.6 per cent of senior secondary teachers. Based on this data, most respondents can be classified as experienced, senior teachers. Table 4. Teachers working experiences School Level Elementary Junior Secondary Senior Secondary Total 0-3 years N % 6 14.6 8 20.5 4-10 years n % 10 24.2 24 61.5 11-20 years N % 7 17.1 6 15.4 21- 30 years n % 10 24.4 1 2.6 >30 years n % 8 19.5 0 0 Total N 41 39 8 29.6 12 44.4 6 22.2 0 11 1 3.7 27 22 20.6 46 43.0 19 17.8 11 10.3 9 8.4 107 Since the participants surveyed and interviewed were predominantly experienced teachers, it is assumed they are more likely to know the state of ICT than the newer teachers and as such the information gathered represents the real conditions in Bojonegara Sub District. Data provided by teachers, regarding the ICT conditions in Bojonegara Sub District was considered the primary data but this was cross checked and validated by the researcher visiting and documenting (photos and video) ICT infrastructure, facilities and resources in the area and schools. As this study is concerned with ICT, the number of teachers trained in relation to ICT was explored as this relates to teachers ICT skills. As can be seen only a very small percentage of teachers and all from the senior secondary school had been trained in the ICT field. Table 5. Teachers experience on attending ICT related training Never 1-2 times 3-5 times > 5 times School Level N % n % N % N % Elementary 30 73.2 8 19.5 1 2.4 2 4.9 Junior 9 23.1 17 43.6 12 30.8 1 2.6 Secondary Senior 11 40.7 13 48.1 2 7.4 1 3.7 Secondary Total 50 46.7 38 35.5 15 14.0 4 3.7 Total N 41 39 27 107 Although the official educational background of teachers is good, their training, attending or participating in ICT related training is still low. In total only 3.7 per cent teachers have more than 5 training experiences related to ICT, and only 14 per cent of them have 3 to 5 training experiences related to ICT. This finding suggests that the teachers ICT skills may be poor. If there are many teachers in Bojonegara Sub District with good ICT skills, it would be because of their self-interest and experiences with ICT, and not because of the official training conducted by schools or the Education Office. This finding in fact can be used as a basis for recommending the Education Office conduct a number of ICT related training programs for teachers in Bojonegara Sub District, Banten Province, Indonesia. Teachers’ Real and Perceived of ICTs Supported-Situation … Fathurrohman, Porter, & Worthy 18 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Findings of ICTs Supported-Situation There are various levels at which ICT conditions can be examined, in the sub district, at the school, and the home levels. 1. In the sub District In recent years, a substantial amount of ICT infrastructure and facilities have been developed in this region of Banten Province. Base transceiver station (BTS) towers, constructed in this region, now deliver wireless signals for hand phones and internet connection, enhancing communication both within and outside the school district. 2. At the school level In elementary schools, computer and related facilities are only used for administrative purposes, such as writing letters or administration reports; they are not used for teaching and learning. The equipment is supplied by the Education Office for school administration purposes not to support teaching and learning processes. All elementary schools visited during observation had TV and related electric equipment (such as CD/DVD players, refer Figure 7 for a typical set up). In junior and senior high schools, observation revealed that 66.7 per cent of the sampled schools had one computer laboratory for teaching and learning. Teachers through the questionnaire revealed a similar percentage of schools containing at least one laboratory. Where computer laboratories are used the usage is not high. There is no computer laboratory use for teaching and learning at elementary school. Results of interview and photo documentation confirmed junior and secondary schools usually have a computer laboratory. According to the interviews, state schools received these computers, electronic resources, and related devices from the government. Private schools receive a donation from students’ parents or grants from their founders. At least until these data were collected, although there are many private companies (some of them are foreign companies and manufacturing factories) in Bojonegara Sub District, schools in this area received neither grant or benefit from these companies for ICT infrastructure, facilities, and resources. There is an opportunity for the government to encourage companies as part of their Corporate Social Responsibility (CSR) to consider giving educational grants to schools, by giving ICT facilities to schools or by providing support in the form of scholarships for poor students. CSR is mandated for good corporate governance, and should be implemented by these companies, because they are located near enough to schools and rich enough to do it. At elementary schools, computers, printers, and other electronic equipment are only for use for administrative purposes, not for teaching and learning. Students learn in the class with conventional teaching and learning processes involving teacher instruction at the whiteboard. However at junior and secondary school, the computer laboratories are available. Although in some schools the number of computers in the computer laboratory is not more than 20 units, whereas class size is 30 to 40 students, still their condition is good and they can be used for teaching and learning. Observation indicated there was internet access (wireless) in the junior and senior high schools, and although the speed of the internet access was not good, it was still convenient to use for accessing web sites for searching and downloading learning resources from the internet. The researcher also observed one teacher accessing a IJEME, Vol. 1, No. 1, March 2017, 11-24 IJEME ISSN: 2549-4996 19 social network account during the visit using a notebook connected to the internet, a further indication of use of the internet. Notebooks (portable computers) or projectors are available at each school level. All junior secondary teachers had access to a notebook and computer in their school, however approximately 30 per cent of teachers in elementary and senior secondary school reported that they did not have access to these notebooks and projectors. This may have happened due to limitations in the number of notebooks and projectors. Table 6. The use of computer notebooks or projectors for teaching and learning No 1-2 times 3-5 times 6-10 times Total per month per month per month School Level N % N % n % n % N Elementary 29 100.0 0 0.0 0 0.0 0 0.0 29 Junior 22 71.0 5 16.1 3 9.7 1 3.2 31 Secondary Senior 24 92.3 2 7.7 0 0.0 0 0.0 26 Secondary Total 75 87.2 7 8.1 3 3.5 1 1.2 86 In terms of internet access, this facility is only available to teachers in junior and senior secondary schools. Observation confirmed this internet access with the researcher detecting the wifi signal and observing teachers connect their own notebooks to the internet using school provided connections. Table 7. School levels and the teachers’ use of internet for teaching and learning No 1-2 times 3-5 times 6-10 times Total per month per month per month School Level N % N % N % n % N Elementary 41 100.0 0 0.0 0 0.0 0 0.0 41 Junior 26 74.3 7 20.0 1 2.9 1 2.9 35 Secondary Senior 24 88.9 2 7.4 0 0.0 1 3.7 27 Secondary Total 91 88.3 9 8.7 1 1.0 2 1.9 103 Teachers reported using their personal mobile phone in the school for internet access. The interviews, in accord with the questionnaires also reveal that several teachers have an internet connection from their own mobile phone and frequently access the internet using them. They know that the internet is useful and they know how to use it. In addition, 18.8 per cent of teachers frequently access the internet using their mobile phones more than five times a day in the school. Table 8. School level and number of teacher internet accesses with mobile phones No 1-2 times 3-5 times > 5 times Total per day per day per day School Level n % N % N % n % n Elementary 12 85.7 2 14.3 0 0.0 0 0.0 14 Junior Secondary 1 5.9 8 47.1 1 5.9 7 41.2 17 Senior Secondary 6 35.3 6 35.3 3 17.6 2 11.8 17 Total 19 39.6 16 33.3 4 8.3 9 18.8 46 Teachers’ Real and Perceived of ICTs Supported-Situation … Fathurrohman, Porter, & Worthy 20 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Fifty-one percent of teachers used their mobile phones to access the internet. These finding suggest that teachers are familiar with the internet, and are thus able and interested in accessing internet although they were on working days at school. With these skills and internet access at least at the high school level, it is possible that teachers could be supported to use resources on the internet for teaching and learning. The usability of ICT at schools depends upon it being maintained and serviced during technical problems that can occur during use. For this reason, there should be one or more IT staff, whether officially employed in the school or by another company, such as computer service shop. There is no IT staff at elementary schools nor did visits by IT professionals, while in total 56.1 percent teachers of junior and senior secondary staff state that IT staff is available. In junior and secondary highschool, 24.2 per cent of teachers indicated that IT staff visit only 1 or 2 times per month, while 31.8 cent stated that IT staff visit schools more than 10 times per month. 3. At the teachers’ home In contrast to the availability of technology in schools, the highest percentage of teachers who have computers at home are the elementary teachers (82.5%) with the least number of computers at home reported by junior secondary (72.2%) and then the senior secondary teacher (64%). Further results, in regard to available computer and related devices and services. Table 9. Available computer and related devices & services Elementary Junior Secondary Senior Secondary At home N % N % N % Computer 33 82.5 26 72.2 16 64.0 Printers & Scanners 28 70.0 20 55.6 14 56.0 Internet Access 1 2.5 13 35.1 18 30.8 Personal Website or Blog 0 0.0 2 6.5 5 20.8 Phone internet Access 6 15.0 27 79.4 12 63.2 IT assistance 27 67.5 18 52.9 18 81.8 The number of teachers who use their own computer at home for preparation of teaching and learning is low particularly for elementary teachers, 96.4 percent of whom never use the home computer for teaching and learning. Teachers’ use of their own computers for teaching and learning includes the preparation of learning resources, learning designs, tasks, downloading learning materials from internet. The findings show that the teachers of senior and junior secondary schools are more familiar with the use of computer for teaching and learning than teachers in elementary schools. The use of computer is not in term of programming. At home Table 10. The use of computers for teaching and learning Junior Senior Elementary Total Secondary Secondary n % N % n % % Computer Never 1-2 times per month 3-5 times per month 6-10 times per month > 10 times per month 27 1 0 0 0 IJEME, Vol. 1, No. 1, March 2017, 11-24 96.4 3.6 0.0 0.0 0.0 6 8 5 1 0 30.0 40.0 25.0 5.0 0.0 1 5 1 1 5 7.7 38.5 7.7 7.7 38.5 55.7 23.0 9.8 3.3 8.2 IJEME ISSN: 2549-4996 21 In accord with the questionnaire, those staff interviewed confirmed that the majority of teachers have their own computer at home. Teachers stated that their computer at home is used by their family, usually their children, for many purposes such as for playing games, watching movies, and other fun activities. The price of a set of desktop computer today, in Indonesia normally priced Rp4,500,000 to Rp9,000,000 or around $300 to $600 (all $ means Australian Dollar). Some teachers also indicated that they have a computer notebook, in Indonesia normally priced around Rp4,500,000 to Rp30,000,000 or around $300 to $2,000. This range of prices is affordable for many public servant teachers whose base annual salary for new teachers with no professional teaching experience or service is Rp28,423,200 and senior teachers with 32 years or more professional teaching experience or service is Rp65,026,000 per year. This range is lower than annual salary of teachers in some countries, for example in Australia, fouryear trained teacher start on salary $59,706 and the most experienced classroom teachers earning $89,050. For many public servant teachers in Indonesia, the prices of computer and related equipment are affordable. Approximately 20 per cent teachers bring the computer notebook to the school for their own purposes. Although the number of teachers who have printer or scanner is less than those who own computers, in total the 61.4 per cent of teachers have their own printer or scanner. Despite the large number of teachers (n=55, 61.4%), who have their own printer or scanner, only 31.8 (n=23) percent use it to support teaching and learning. These and other findings suggest there is a potential to support teachers’ use of their home computer for teaching and learning. The number of teachers who have and use internet access and phone internet access for teaching and learning is low particularly for elementary teachers who never use internet access and phone internet access for teaching and learning. Teachers’ use of internet access and phone internet access for teaching and learning are for activities such as downloading learning materials from internet, searching educational articles to learn how to improve their teaching and learning. The findings show that the teachers of senior and junior secondary schools are more familiar with the use of computer for teaching and learning than teachers in elementary schools. Twenty-one per cent of teachers, all but one secondary teacher has internet access at home. The teachers of senior and junior secondary schools are more familiar with the use of internet for teaching and learning with none of the elementary teachers using their home internet for teaching and learning purposes. Of the teachers who have the internet at home only around 25 per cent, use it for teaching and learning purposes, however it is possible that teachers can be supported to take advantage of the resources on the internet for teaching and learning. At home, 48.4 per cent of teachers access the internet using their mobile phone, and the number of them who frequently access the internet using mobile phone is also large, up to 59.1 per cent. Only a small number (7.4%) of teachers have a personal web site or blog, however the use of personal web sites or blogs for teaching and learning is negligible. Only three teachers (15%) of those who had a blog or personal website used these for school purposes. At home IT assistance No schedule 1-2 times per week Everyday available Table 11. IT assistance at home Elementary Junior Secondary Senior Secondary N % n % n % 14 77.8 14 70.0 13 76.5 4 22.2 6 30.0 3 17.6 0 0.0 0 0.0 1 5.9 Total n Total % 41 13 74.5 1 23.6 1.9 Teachers’ Real and Perceived of ICTs Supported-Situation … Fathurrohman, Porter, & Worthy 22 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 In regard to mathematical learning resources and learning designs. Teachers and students can freely use Buku Sekolah Elektronik (BSE) that is Electronic School Books. These electronic books are available in PDF format. There are several books that can be used by students and teachers from Class I, the first stage of formal education in Elementary Schools to Class XII, the last stage of Senior High School before university level. The Ministry of Education and Culture owns the copyright to these books and the contents are evaluated and monitored by the Ministry of Education and Culture. Teachers revealed in the interviews that they did not use them for teaching and learning. Some did not have or know about these books. Teachers might not consider the information in these books to be effective. Several senior and junior secondary schools have their own notebooks. According to interview, teachers stated that around 20 percent of teachers in their schools bring it to the school for their own purposes. Teachers also explained that they and their fellow teachers use the internet to download educational content for use in teaching and learning. The internet is sometimes their alternative source of teaching materials as they locate many internet available resources that are useful for their teaching. Exploring Possibilities for Mathematics Teaching and Learning Teachers were asked about the possibility of them implementing technologybased mathematical learning resources, through the internet or online platform in their school or class. As indicated in Table, 53.1 percent of junior secondary teachers and 23.5 percent teachers of senior secondary schools believe that the implementation is possible, however all, elementary school teachers considered that the implementation of technology-based mathematical learning resources “may be possible but it would be difficult”. These results are confirmed through interview with teachers. However teachers stated that the condition of ICT in their schools is problematic when it comes to the implementation of technology-based teaching and learning. One of teachers in the interviews stated that in general many teachers have good skills and are interested in implementing technology in the teaching and learning, including the integration of the internet to improve the quality of teaching and learning, however ICT infrastructure and facilities, such as the quantity and quality of available computers for teaching and learning, the lack of sufficient supporting equipment, such as projectors, the quality of internet access, software, and financial support required would make proper implementation of technology-based teaching and learning difficult. Other teachers interviewed made similar statements. In general teachers stated that poor ICT conditions is the main problem for the implementation of technology-based teaching and learning in Bojonegara Sub District, although they believe that the teachers have good computer skills, and are able to operate computers and other facilities for teaching and learning. The observer/researcher saw that one of the problems related to a mindset of teachers that the computers, related equipment, and internet access must be available before the implementation of technology-based teaching and learning. This meant that teachers did not try to or know how to optimize their use of the limited ICT for teaching and learning under the current ICT conditions. There were possibilities to support teaching and learning, or in general to improve the learning experiences of students using the current condition of ICT even though the ICT conditions in their school were poor. IJEME, Vol. 1, No. 1, March 2017, 11-24 IJEME ISSN: 2549-4996 23 The researcher/observer was seeking ways maximize the impact of ICT on teaching and learning given the current condition of ICT infrastructure and facilities, such as computers, the related equipment and internet access in schools and teachers’ homes, to improve students’ learning experiences, especially in mathematics, without having to request an additional infrastructure, facilities, and resources that would be costly for schools or government. This thinking lead to the development of tools related to the technology-based teaching and learning, appropriate to current ICT conditions, but tools that could contribute toward improving learning experiences. When examining the desire or willingness for teachers to create and share both resources and learning designs, seventy four per cent of teachers indicated that they had experienced creating or modifying learning resources. For learning designs, only 19.3 percent of teachers have tried to initiate learning designs in the classroom or school. In the case of sharing of learning resources and learning designs, as revealed in Table, 27.4 percent of teachers wish to share their own learning resources, compared to 93.3 percent of teachers who wish to share their learning design. The researcher also asked teachers about whether they had experiences creating or modifying digital learning resources for use through the internet or online. With respect to the use of learning design, 86.4 percent of teachers stated that they wish to use other teachers mathematical learning design, with only a few teachers experienced in creating digital mathematical learning resources. Teachers also wanted to use other teachers’ learning designs, with few teachers creating their own designs; therefore the circulation of a good learning design between teachers appears warranted. This finding became the foundation for the mapping learning design for circulation by one teacher to other teachers and complementing this the mapping of internet accessible learning resources for use by teachers. The above condition is suitable for condition where teachers directs the pace and sequence. The implementation can consider result of empirical study on how the implementation of technology will contribute to students performance Fathurrohman, Porter, and Worthy (2012) This condition also supports the condition where the instruction of teaching and learning, related to presentation for in class sessions, including the target on specific skill development. Based on above discussion, it is suggested that the Teachers-Centered Learning with Technology is the most appropriate method of technology-based learning to be implemented in Bojonegara Sub District, Banten Province Indonesia CONCLUSION Based on the results of observation, several important findings about the physical state and teachers’ awareness of ICT infrastructures, facilities, and resources were gathered. Teachers’ access and use of the internet to gather educational content for use in teaching and learning has been explored. The internet is one of the sources of learning resources. Since the teachers wish to use other teachers learning designs and have little experience in creating learning designs, it is appropriate to develop tools that assist the circulation of good learning designs between teachers. TeachersCentered Learning with Technology is the most appropriate method of technologybased learning to be implemented. Teachers’ Real and Perceived of ICTs Supported-Situation … Fathurrohman, Porter, & Worthy 24 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 REFERENCES Agostinho, S. (2009). Learning Design Representations to Document, Model, and Share Teaching Practice. In Lockyer, L., et al. (2009). Handbook of Research on Learning Design and Learning Objects; Issues, Application, and Technologies. Vol. 1. New York: Information Science Reference. De Vaus, D., A. (2002). Surveys in Social Research (5th Edition). Crows Nest: Allen & Unwin. Fathurrohman, M., & Porter, A. (2012). Addressing the needs of a developing nation: electronic maps of mathematical learning resources accessible via the internet. Journal of Computers in Mathematics and Science Teaching, 31(4), 337-362. Fathurrohman, M., Porter, A., & Worthy, A., L. (2014). Comparison of performance due to guided hyperlearning, unguided hyperlearning, and conventional learning in mathematics: an empirical study. International Journal of Mathematical Education in Science and Technology, 45(5), 682-692. InfoDev. (2007). Survey of ICT for Education in Africa. World Bank. InfoDev. (2009a). Survey of ICT for Education in Caribbean. World Bank. InfoDev. (2009b). Various Country Study Reports and Case Study Working Papers (Argentina, Brazil, Kenya, Korea, Morocco, Nigeria, Philippines, Peru, Senegal, Sudan, Tunisia, Uganda, and Vietnam) in regard to technology and or education. World Bank. InfoDev. (2010). Survey of ICT for Education in India and South Asia Countries. World Bank. International Development Research Centre. (2010). Digital Review of Asia Pacific. Oxford Dictionaries. (2011). Available at: http://oxforddictionaries.com/ Ross, T.W., & Bailey, G.D. (1995). Technology-Based Learning; A Handbook for Principals and Technology Leaders. New York: Scholastic Inc. World Economic Forum. (2016). The Global Competitiveness Report. Geneva. IJEME, Vol. 1, No. 1, March 2017, 11-24 International Journal on Emerging Mathematics Education (IJEME) Vol. 1, No. 1, March 2017, pp. 87-98 P-ISSN: 2549-4996, E-ISSN: 2548-5806, DOI: http://dx.doi.org/10.12928/ijeme.v1i1.5698 Using the 5E Learning Cycle with Metacognitive Technique to Enhance Students’ Mathematical Critical Thinking Skills 1Runisah, 2Tatang Herman, 2Jarnawi Afgani Dahlan 1Wiralodra University, Jl. Ir. H Juanda KM. 03, Karanganyar, Indramayu, Jawa Barat 45213 2 Universitas Pendidikan Indonesia, Jl. Dr. Setiabudhi, No. 229, Bandung 40154 Email: [email protected] Abstrak Penelitian ini bertujuan untuk mendeskripsikan peningkatan dan pencapaian kemampuan berpikir kritis matematis siswa yang menerima Learning Cycle 5E dengan teknik Metakognitif, Learning Cycle 5E, dan pembelajaran konvensional. Penelitian ini menggunakan metode eksperimen dengan desain kelompok kontrol pretest-posttest. Populasi adalah siswa SMP di Indramayu, Indonesia. Sampel terdiri dari tiga kelas siswa kelas VIII dari sekolah level tinggi dan tiga kelas dari sekolah level sedang. Hasil Penelitian mengungkapkan bahwa dilihat dari siswa secara keseluruhan, peningkatan dan pencapaian kemampuan berpikir kritis siswa yang menerima Learning Cycle 5E dengan teknik Metakognitif lebih baik dari siswa yang menerima Learning Cycle 5E dan pembelajaran konvensional. Kemampuan berpikir kritis matematis siswa yang menerima Learning Cycle 5E lebih baik dari siswa yang menerima pembelajaran konvensional. Tidak ada pengaruh interaksi antara model pembelajaran dan level sekolah terhadap peningkatan dan pencapaian kemampuan berpikir kritis matematis siswa. Kata Kunci: kemampuan berpikir kritis matematis, learning cycle 5E, teknik metakognitif Abstract This study aims to describe enhancement and achievement of mathematical critical thinking skills of students who received the 5E Learning Cycle with Metacognitive technique, the 5E Learning Cycle, and conventional learning. This study use experimental method with pretest-posttest control group design. Population are junior high school students in Indramayu city, Indonesia. Sample are three classes of eighth grade students from high level school and three classes from medium level school. The study reveal that in terms of overall, mathematical critical thinking skills enhancement and achievement of students who received the 5E Learning Cycle with Metacognitive technique is better than students who received the 5E Learning Cycle and conventional learning. Mathematical critical thinking skills of students who received the 5E Learning Cycle is better than students who received conventional learning. There is no interaction effect between learning model and school level toward enhancement and achievement of students’ mathematical critical thinking skills. Keywords: mathematical critical thinking skills, 5E learning cycle, metacognitive technique. How to Cite: Runisah, Herman, T., & Dahlan, J.A. (2017). Using the 5E learning cycle with metacognitive technique to enhance students’mathematical critical thinking skills. International Journal on Emerging Mathematics Education, 1(1), 87-98. http://dx.doi.org/10.12928/ijeme.v1i1.5698. INTRODUCTION Various problems in life aspect occur in 21st century. To overcome this problem, critical thinking skills is needed. According to Chukwuyenum (2013), critical thinking had been used as one way to solve the problem in daily life because it involve logical reasoning, interpretation, analysis, and evaluate information so enable us to obtain valid and reliable decision. Based on his idea, someone who has critical thinking skills, will chose problem representation which is most suitable to help solving the problem, Received January 24, 2017; Revised February 24, 2017; Accepted February 27, 2017 88 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 then choosing and using strategy to solve the problem which is backed by reason and proof. Therefore, problem solving and decision which are given will be backed by accurate reason or proof. Critical thinking skills is high order thinking skills, in which the expert defined it in different way. One of them is Ennis (1996) who defined, “Critical thinking is reasonable and reflective thinking that is focused on deciding what to believe or do”. Reflective means consider or thinking again everything faced before making decision. Reasonable means that all beliefs, views, or everything done which are backed by appropriate proof or reason. Critical thinking is needed in various domains, included mathematics. According to Balcaen and Klassen (2007), critical thinking in mathematics is involvement of thinking through mathematical problem and make reasonable assessment about strategy, approach and solution. In accord with that opinion, Glazer (2001) explained that critical thinking in mathematics is ability and disposition to involve prior knowledge, mathematical reasoning, and cognitive strategy to generalize, prove or evaluate unfamiliar mathematical situation reflectively. Based on that definition, critical thinking in mathematics can be defined as ability to integrate prior knowledge, mathematical reasoning, and problem solving strategy to solve mathematical problem reflectively. Furthermore, according to Innabi (2003) critical thinking aspect related with learning material which comprise: concept, generalization, skill, algorithm and problem solving. `The result of study shows that mathematical critical thinking skills (MCTS) development can enhance mathematics achievement (Jacob, 2012; Chukwuyenum, 2013). Similarly, critical thinking skills will encourage students to think independently and solve problem in school or in the context of everyday life (Jacob, 2012). Critical thinking is not limited to reflection, inference, and synthesis the information, enable individual to make reasonable assessment not only in class but in daily life (Beaumont, 2010). Thus, the critical thinking skills, which are developed through learning activities in the school, will be useful for solving various problems, whether the problems are directly related to learning activity or problems are in their daily lives. In other words, through the critical thinking skills, students will be able to consider or choose the important information that can be used to solve problems or to make a reasonable decision. By viewing how important the development of critical thinking skills is, so that critical thinking development become curriculum agenda in the world, particularly in Indonesia. In Kurikulum Tingkat Satuan Pendidikan (KTSP) as stated in Permendiknas No.22 (2006) which is applied in Indonesia, it is implied that mathematics need to be given to students to equip them with logical, analytical, systematical, critical and creative thinking. Furthermore in 2013 curriculum as stated in Permendikbud No. 64 (2013), critical thinking is also stated in learning mathematics. Even though critical thinking skills become one goal in learning particularly in mathematics, several result of study showed that this ability is still low. Hiebert (Lithner, 2008) reported that generally students still use thinking based on memorization than doing reasoning process in solving mathematical problem. The result of Trends in International Mathematics and Science Study (TIMSS) in 2007 and 2011 showed that average score of mathematics subject achievement in 2011 is in 38th rank from 42 participating countries. In TIMSS 2011 students are involved in various cognitive processes to solve the problem (Mullis, et al., 2012). Furthermore, Runisah IJEME, Vol. 1, No. 1, March 2017, 87-98 IJEME ISSN: 2549-4996 89 (2015) based on study toward 8th grade in Indramayu city, Indonesia concluded that students’ mathematical critical thinking skills is still low. One factor which affect the lack of students’ mathematical critical thinking skills is learning process. In Indonesia, teaching practice focused on material content and ignore students’ thinking ability development (Rohaeti, 2010). From inquiry result of Balitbang Depdiknas in 2007, implementation of learning in Indonesia generally still use lecture and ask and answer methods (Balitbang Depdiknas, 2007). Activities can be done to enhance critical thinking skills in mathematics such as comparing, making conjecture, making induction, making generalization, making specialization, making classification, making deduction process, making visualization, ordering, making prediction, making validation, proving, analyzing, evaluating and making pattern, and determining functional relation among variables (Appelbaum, 1999). Futhermore, Beaumont (2010) stated that to enhance students’ critical thinking skills, exercises in the form of task need to be given which require high reasoning to solve it such as task of observing, identifying assumption, challenging material to be understood, task of interpreting, task of discovering and investigating, task of analyzing and evaluating, and task of making decision. Therefore, to enhance mathematical critical thinking skills non routine tasks or problem need to be given which require high reasoning to solve it. One of learning potentially to develop critical thinking skills is the 5E Learning Cycle with Metacognitive technique (LCM). The 5E Learning Cycle (LC) is developed by researcher team of Biological Science Curriculum Study leaded by Bybee. According to Bybee, et al. (2006) LC is influenced by Herbart psychology, John Dewey and Jean Piaget thinking. Piaget with his constructivism principle viewed that knowledge is not a set of fact, concept or rule which are ready to be transferred by teacher. Students should construct that knowledge and give meaning through various experiences in learning. According to Bybee, et al. (2006), LC has five stages namely: engage, explore, explain, elaborate and evaluate. In engage stage, teacher access students’ prior knowledge and help them to involve in new concept which encourage students interest to learn. In explore stage, students are involved in concept exploration activity to produce new ideas. In explain stage, students explain conceptual understanding or process skill which is obtained in earlier stage. In evaluate stage, assessment is done toward students’ understanding and ability and give opportunity to teacher to evaluate students progress to achieve educational aim. Those five stages can trigger students’ critical thinking skill, because it involve prior knowledge, non routine situation, reasoning, cognitive strategy, and involve students in discussion to do exploration. This is in accord with Appelbaum (1999) and Beaumont (2010) argument that condition to critical thinking in mathematics beside should contain non routine situation, it should also use prior knowledge, reasoning, and cognitive strategy. Furthermore, according to Slavin (2011), critical thinking teaching which is effective depend on classroom determination that encourage different point of view acceptance and free discussion. Meanwhile, according to Ergin (Tuna & Kacar, 2013), LC involve high thinking order skill by stimulating students to explore. LC transmit critical thinking skill to students. The 5E Learning Cycle with Metacognitive technique (LCM) is learning model that integrate directly Metacognitive technique in every stage of LC. Metacognition is a term introduced by Flavell in 1976. Flavell (Lioe et al., 2006) stated that metacognition is one’s consciousness about his/her cognitive process and independency to achieve Using the 5E learning cycle with metacognitive technique to enhance students’ … Runisah, Herman, & Dahlan 90 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 the goal. One of Metacognitive technique is self-asking. In this study, question which is made focus on three categories which are adopted from Beeth (Mittlefehldt & Grotzer, 2003) namely intelligibility, wide-applicability, and plausibility. In first category that is intelligibility, the question asked is, “Am I able to understand the concept I learned?” In second category, that is wide-applicability, the question asked is, “What concept that can be used to solve this problem?” or, “Is the concept I learned can be used to solve the problem in another domain or in daily life?” In third category, that is plausibility, the question asked is: “Is problem solution I made can be reliable?”. From the above description, the use of LCM has potencies to develop students' critical thinking skills. However, the application of the model may not have the same effectiveness when it is applied to students who have different academic abilities. In other words, the use of LCM is probably more effective for a particular group rather than it is used in another group of students who have different academic abilities. This is in line with the opinion of Glazer (2001) as it is already described above that in critical thinking, reasoning, cognitive strategies, and the students' prior knowledge play an important role. Therefore, the analysis of the effect of the interaction between learning model and school level toward critical thinking skills is required. It is useful to determine the level of school where the model can be used more effectively, because the two level different schools have difference in a student's academic ability There have been numerous studies on several approaches that is different from the 5E Leaning Cycle to enhance students’ mathematical critical thinking skills (Rohaeti, 2010; Noer, 2010; Kurniati, Kusumah, Sabandar, & Herman, 2015; Yumiati, 2015; Firdaus, Kailani, Nor, & Bakry, 2015). Moreover some studies have been done on the use of the 5E Learning Cycle to enhance students’ mathematical critical thinking skills (Erlian, 2009; Fatimah, 2012; Sofuroh, Masrukan, & Kartono, 2014; Kadarisma, 2015). Meanwhile some studies on Metacognitive empowermen has been done (Schraw (Toit & Kotze, 2009); Camahalan, 2006; Ozcan & Erktin, 2015). However, only a bit studies on the use of the 5E Leaning Cycle with Metacognitive technique to enhance students’mathematical critical thinking skills. Therefore, the present study purposes at examining the use of the 5E Learning Cycle with Metacognitive technique to enhance mathematical critical thinking skills of junior high school students in one of cities in west Java, Indonesia. This research was conducted in Indramayu west Java by considering the problem that is identified by the preliminary study. Preliminary study involved 33 junior high school students (Runisah, 2015) reported that the mean score of MCTS test of students is only 5.19 of the ideal maximum score of 16. This result shows that the critical thinking skills of students in mathematics were still low. Every question on the MCTS test was related to aspects of critical thinking with mathematical content, it includes concepts, generalization, and problem solving RESEARCH METHOD This study is a experimental with pretest-postest control group design (Ruseffendi, 2005) described as follows: A O X1 O A O X2 O AOO IJEME, Vol. 1, No. 1, March 2017, 87-98 IJEME ISSN: 2549-4996 91 Information: A : The selection of a random sample of classes at population X1: The application of The 5E Learning Cycle with Metacognitive technique (LCM) X2 : The application of The 5E Learning Cycle (LC) O: Mathematical Critical Thinking Skills (MCTS) test (pretest-postest). Population and Sample Population in this study is junior high school students in Indramayu city, West Java Province, Indonesia. The sample is eighth grade students amounting to 173 students from two school level, classified as high amounting to 83 students and medium level amounting to 90 students. The selection of that school is done randomly from all Junior High Schools in Indramayu city. Three classes are selected randomly from each school level, one class are taught by LCM, one class are taught by LC, and one class are taught by CL. The determination of school level, based on accreditation score which is valid until the year 2016. Instruments Instrument used in this study consist of mathematical critical thinking skills (MCTS) test, Mathematical Prior Ability (MPA) test, and observation sheet. MCTS test consist of 10 items with ideal maximum score is 40. MCTS test is given to students before and after learning is implemented. MCTS test material is tailored with material given in the time of study, that is material of 8th semester 1 is in accord with curriculum used. Before used, the experts consider MCTS test to fulfill face and content validity. Then try out test is done in limited scale. After being improved, instrument is tested in wide scale. Based on test result, it is obtained that test is valid and reliable with reliability coefficient r = 0.84 and according to Creswell (2012) it is in high category. Meanwhile, material of MPA test was adjusted to the subject matter of Mathematics, which has been studied in the previous semester refered to the curriculum. Based on analysis, it is obtained that test is valid and reliable with reliability coefficient r = 0.83 for MPA test, with objective form and r = 0.64, for MPA with analytical test. MPA test is used for further convince that the MPA at the high school level is better than MPA at medium school level. The magnitude of students’ MCTS achievement is obtained from MCTS posttest score. The formulation developed by Meltzer (2002) is used to calculate the magnitude of enhancement. Whereas the calculation result of gain is interpreted by using classification of gain from Hake (1998). RESULTS AND DISCUSSION This part describes the result of the study and its discussion which is related to the relevant studies and theories. The Enhancement of Students’ MCTS Base on normality test, data were not distributed normally. Therefore KruskalWallis test is used to test mean difference of MCTS enhancement which is presented in Table 1. Using the 5E learning cycle with metacognitive technique to enhance students’ … Runisah, Herman, & Dahlan 92 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Table 1. Summary of Mean Difference Test of MCTS Enhancement School g Group Chi-Square Sig. Ho Level High LCM; LC; CL 0.68; 0.53; 0.38 30.21 0.000 rejected Medium LCM; LC; CL 0.57; 0.46; 0.30 35.07 0.000 rejected Total LCM; LC; CL 0.62; 0.49; 0.34 57.37 0.000 rejected LCM : The 5E Learning Cycle with Metacognitive Technique LC : The 5E Learning Cycle CL : Conventional Learning. Based on Table 1, it is found out that probability (significance) value is less than significance degree α = 0.05, thus H0 is rejected. Thus in terms of overall and for each school level, it means that at least there is one group who has mean gain which is different from another group. Further, based on Multiple Comparison Between Treatments test at significance degree α = 0.05, in terms of overall and in high school level, MCTS enhancement of students who are taught by LCM is better than students who are taught by LC and students who are taught by CL. MCTS enhancement of students who are taught by LC is better than students who are taught by CL. In medium school level, there is no difference of MCTS enhancement between students who are taught by LCM and students who are taught by LC. However, MCTS enhancement of students who are taught by LCM and students who are taught LC is better than students who are taught by CL. Interaction Effect between Learning Model and School Level toward Students’ MCTS Enhancement The Adjusted Rank Transform test (Leys and Schumann, 2010) is done to find out interaction effect between learning model and school level toward students’ MCTS enhancement. From the calculations, the value of F = 1.20 with a probability value 0.304. Thus, it can be concluded that there is no interaction effect between learning model and school level toward students’ MCTS enhancement. The Achievement of Students’ MCTS The achievement of students’ MCTS is determined based on posttest score. Further, percentage of students’ MCTS achievement can be seen in Figure 1. IJEME, Vol. 1, No. 1, March 2017, 87-98 IJEME ISSN: 2549-4996 93 Figure 1. The Achievement of Students’ MCTS From Figure 1, it can be found out that in terms of overall and for all school levels, MCTS achievement of LCM group is higher than LC group and CL group, MCTS achievement of LC group is higher than CL group. Further, based on normality test, data were not distributed normally. Therefore Kruskal Wallis test is used to test mean difference of MCTS achievement which is presented on Table 2. Table 2. Summary of Mean Difference Test of MCTS Achievement School Level Group Posttest Chi-Square Sig. High LCM; LC; CL 29.31; 24.30; 19.03 24,59 0.000 Medium LCM; LC; CL 25.57; 21.20; 16.57 24.29 0.000 Total LCM; LC; CL 27.30; 22.84; 17.80 44.01 0.000 LCM : The 5E Learning Cycle with Metacognitive Technique LC : The 5E Learning Cycle CL : Conventional Learning. Based on Table 2, it is found out that probability (significance) value is less than significance degree α = 0.05, thus H0 is rejected. Thus in terms of overall and for each school level, it means that at least, there is one group who has mean gain which is different from another group. Further, based on Multiple Comparison Between Treatments test at significance degree α = 0.05, in terms of overall and from high and medium school level, MCTS achievement of students who are taught by LCM is better than students who are taught by LC and students who are taught CL. MCTS achievement of students who are taught by LC is better than students who are taught by CL Using the 5E learning cycle with metacognitive technique to enhance students’ … Runisah, Herman, & Dahlan 94 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Interaction Effect between Learning Model and School Level toward Students’ MCTS Achievement The Adjusted Rank Transform test (Leys and Schumann, 2010) is done to find out interaction effect between learning model and school level toward students’ MCTS achievement. From the calculations, the value of F = 0.421 with a probability value 0.667. Thus, it can be concluded that there is no interaction effect between learning model and school level toward students’ MCTS achievement. In terms of overall, MCTS enhancement of students who are taught by LCM is 0.62, whereas students who are taught by LC is 0.49 and students who are taught by CL is 0.34. That enhancement is in medium category based on classification of Hake. MCTS enhancement of students is supported by its achievement. MCTS achievement of students who are taught by LCM is 27.30 or 68.3% from ideal maximal score. MCTS achievement of students who are taught by LC is 22.84 or 57.1% from ideal maximal score. Meanwhile, MCTS achievement of students who are taught by CL is 17.80 or 44.5% from ideal maximal score. Furthermore, based on statistic test result MCTS enhancement and achievement of students who are taught by LCM is better than students who are taught by LC and students who are taught by CL. MCTS enhancement and achievement of students who are taught by LC is better than students who are taught by CL. Based on statistic test result, the result obtained in terms of overall and students from high school level has similarity namely MCTS enhancement and achievement of students who are taught by LCM is better than students who are taught by LC and students who are taught by CL. MCTS enhancement and achievement of students who are taught by LC is better than students who are taught by CL. In medium school level, it is concluded that there is no difference of MCTS enhancement between students who are taught by LCM and students who are taught LC in significance degree of 0.05. Nevertheless, MCTS achievement of students who are taught by LCM is higher than students who are taught by LC, MCTS enhancement of students who are taught by LCM is 0.57 and for students who are taught by LC is 0.46. Meanwhile, MCTS enhancement of students who are taught by LCM and students who are taught by LC is better than students who are taught by CL. Futhermore, MCTS achievement of students who are taught by LCM is better than students who are taught by LC and students who are taught by CL. MCTS achievement of students who are taught by LC is better than students who are taught by CL. In general, the result of study show that LCM is better in facilitating students to develop MCTS than LC and CL, and LC is better in facilitating students to develop MCTS than CL. This is possible because in LC students are involved in learning activity actively through discussion to do activities such as comparing, making conjecture, making generalization, making prediction, making validation, analyzing, evaluating, and determining functional relation among variables. Those activities will develop students’ mathematical critical thinking skills. It is proven what has been stated by several expert (Appelbaum, 1999; Beaumont, 2010; Glazer, 2004). In line with it, according to Slavin (2011) the effective teaching of critical thinking depend on determination of classroom which encourage the acceptance of different point of view and free discussion. Furthermore, Ergin (2012) added that the 5E model is the most effective way to involve students in learning. Students involvement in learning will develop their thinking ability among other critical thinking skills. IJEME, Vol. 1, No. 1, March 2017, 87-98 IJEME ISSN: 2549-4996 95 In LC, students are involved in exploration activity toward concept learned, thus students understanding will become deeper. According to Carpenter (Franke & Kazemi, 2001), when individuals learn with understanding, they can use the knowledge to solve new problems. Meanwhile, in CL, teaching and learning activity is more teacher centered. In CL, teacher give concept which is learned directly, students just receive what is delivered by teacher, then students are given problem exercises. Therefore, in CL learning is dominated by teacher. Thus in CL development of critical thinking skills is lacking. In LCM, besides having strengths contained in LC, students’ metacognition is more empowered compared to LC and CL. Students’ metacognition empowerment is done by guiding student to ask themselves and answer it. Therefore, students will try to realize their thinking process. They will think about their experience toward concept, another domain or relation among concepts. This is strongly support development of critical thinking skills that which will be used in solving the problem. This is in accord with Panaoura and Phillippou (2005) that if someone not aware of his/her process and cognitive ability, we will not be able to improve his/her performance. Furthermore, Schraw and Dennison (Panaoura & Philippou, 2005) concluded that students who are skillful in assessing their Metacognitive and aware of their ability to think are better than students who not aware of their mental system mechanism in solving mathematical problem. This study result is in accord with study result of several expert that the use of the 5E Learning Cycle support students’ critical thinking skills in mathematics (Erlian, 2009; Fatimah, 2012; Sofuroh, Masrukan, & Kartono, 2014). Other than, study result of several expert showed that there is positive influence from constructivism based learning toward enhancement of students’ mathematical critical thinking skills (Rohaeti, 2010; Noer, 2010; Kurniati, Kusumah, Sabandar, & Herman, 2015; Yumiati, 2015; Firdaus, et al. , 2015). This study also show that school level factor has significant effect toward achievement and enhancement of students’ MCTS. For each learning model, students in high school level obtain achievement and enhancement of MCTS which is higher than students in medium school level. In other word, students in high school level get more advantage in achievement and enhancement of MCTS than students in medium school level. This occurs because Mathematical Prior Ability (MPA) of students at the high school level is better than MPA of students at medium school Level. MPA of students is one aspect that support the critical thinking skills in mathematics. This is in line with the opinion of Glazer (2001) that the critical thinking in math involves prior knowledge of the students. This study also find that there is no interaction effect between learning model and school level toward enhancement and achievement of students’ MCTS. This is possible because in one class, each discussion group has relatively the same academic ability. Each group consists of students who have the academic ability of high, medium, and low. This condition causes the application of LCM and LC run smoothly on the high school level as well as at the medium school level. Therefore, LCM and LC can be used in medium and high level school, because in whichever level, MCTS enhancement and achievement of students who are taught by LCM will be higher than students who are taught by LC and CL. MCTS enhancement and achievement of students who are taught by LC will be higher than students who are taught by CL. Using the 5E learning cycle with metacognitive technique to enhance students’ … Runisah, Herman, & Dahlan 96 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 CONCLUSION Based on result study, it can be concluded that in terms of overall and in high school level, MCTS enhancement and achievement of students who are taught by LCM is better than students who are taught by LC and students who are taught by CL. MCTS enhancement and achievement of students who are taught by LC is better than students who are taught by CL. In medium school level, there is no difference of MCTS enhancement between students who are taught by LCM and LC, however MCTS enhancement of students who are taught by LCM and students who are taught by LC is better than students who are taught by CL. Whereas, MCTS achievement of students who are taught by LCM is better than students who are taught by LC and students who are taught by CL. MCTS achievement of students who are taught by LC is better than students who are taught by CL. In terms of overall and students in high school level, MCTS enhancement of students who are taught by LCM, LC, and CL is in medium category. In medium school level, MCTS enhancement of students who are taught by LCM and LC is in medium category, but MCTS enhancement of students who are taught by CL is in low category. Thus the application of LCM and LC has not yet gave maximal effect toward students’ MCTS enhancement. Therefore, further research is needed to examine its cause. There is no interaction effect between learning model and school level toward enhancement and achievement of students’ MCTS. Thus, LCM and LC can be used for high school level and medium school level to enhance students’ MCTS. REFERENCES Appelbaum, P.M. (1999). Eight critical points for Mathematics. Retrieved from http://gargoyle.arcadia.edu/appelbaum/8points.html. Badan Penelitian dan Pengembangan Pusat Kurikulum Departemen Pendidikan Nasional (Balitbang Depdiknas). (2007). Kajian kebijakan kurikulum mata pelajaran matematika. Jakarta: Depdiknas. Balcaen, & Klassen (2007). Teaching critical mathematics thinking (mathematical mindedness). Retrieved from http://libocs.lib.sfu.ca:8087/fedcan/index.php/csse/csse2008/paper/view/437/308 Beaumont, J. (2010). A sequence of critical thinking tasks. TESOL Journal,1(4),427-448. Bybee, R.W., Taylor, J. A., Gardner, A., Scotter P.V., Powel, J.C., Westbrook, A., & Landes, N. (2006). The BSCS the 5E instructional model: Origins and effectiveness and applications executive summary. Retrieved from http://bscs.org/sites/default/files/_legacy/BSCS_the 5E_Instructional_ModelExecutive_Summary_0.pdf Camahalan, F. M. G. (2006). Effects of self regulated learning on mathematics achievement on selected southeast asian children. Journal of Instructional Psychology, 33(3), 194-205. Chukwuyenum, A.N. (2013). Impact of critical thinking on performance in mathematics among senior secondary school students in Lagos state. IOSR Journal of Research & Method in Education, 3(5),18-25. Creswell, J.W. (2012). Educational research: planning, conducting, and evaluating. quantitative and qualitative research, 4nd edition. Boston: Pearson Ennis, R.H. (1996). Critical thinking. United States of America: Prentice-Hall, Inc. Ergin, I. (2012). Constructivist approach based 5E model and usability instructional physics. Latin-American Journal of Physics Education, 6(1), 14 – 20. IJEME, Vol. 1, No. 1, March 2017, 87-98 IJEME ISSN: 2549-4996 97 Erlian, B.P. (2009). Pegaruh penggunaan model daur belajar (learning cycle) terhadap kemampuan berpikir kritis dan sikap siswa kelas XI MA Muallimat NW Pancor pada mata pelajaran Matematika. Jurnal Educatio, 2(4). 73-83. Fatimah, N. (2012). Penerapan model learning cycle 5E dalam mata pelajaran matematika untuk meningkatkan kemampuan berpikir kritis siswa SMA. Retrieved from http://repository.upi.edu/, 2012 Firdaus, Kailani, I. Nor, B.B., & Bakry. (2015). Developing critical thinking skills of students in mathematics learning. Journal of Education and Learning, 9(3), 226236. Franke, M.L., & Kazemi, E. (2001). Learning to teach mathematics focus on student thinking. Theory into practice. 40(2), 102 – 109. Glazer, E. (2001). Using web sources to promote critical thinking in high school Mathematics. Retrieved from http://math.unipa.it/~grim/AGlazer79-84.PDF Glazer, E (2004). Technology enhanced learning environments that are conductive to critical thinking in mathematics: Implication for research about critical thinking on the world wide web. Retrieved from http://www.http://lonstar.texas.net/ ~mseifert / crit2.html. Hake, R.R. (1998). Interactive-engagement versus traditional methods: A sixthousand-student survey of mechanics test data for introductory physics courses. Journal of American Association of Physics Teachers, 66(1), 64 -74. Innabi, H. (2003). Aspects of critical thinking in classroom instruction of secondary school mathematics teachers in jordan. Proceedings of The International Conference, Brno, Czech Republik. 124-129. Jacob, S.M. (2012). Mathematical achievement and critical thinking skills in asynchronous discussion forums. Procedia - Social and Behavioral Sciences, 31 (2012), 800 – 804. Kadarisma, G. (2015). Meningkatkan kemampuan berpikir kritis dan logis matematis serta kemandirian belajar siswa SMP melalui learning cycle 5E dan discovery learning. Thesis. Bandung: Sekolah Pascasarjana Universitas Pendidikan Indonesia. Kurniati, Kusumah, Y.S., Sabandar, J., & Herman, T. (2015). Mathematical critical thinking skills through contextual teaching and learning. IndoMS-JME, 6(1), 5362. Leys, C., & Schumann, S. (2010). A nonparametric method to analyze interactions: The adjusted rank transform test. Journal of Experimental Social Psychology, 46(4), 587-700. Lioe, L.T, Fai, H.K., & Hedberg, J.G. (2006). Students’metacognitive problem solving strategies in solving open-ended problems in pairs. Retrieved from http://conference.nie.edu.sg/paper/new converted/aboo 287.pdf. Lithner. (2008). Research framework for creative and initative reasoning. Educational Studies in Mathematics, 67(3), 255-276. Meltzer, D.E. (2002). The relationship between mathematics preparation and conceptual learning gains in physics: a posible “hidden variable” in diagnostic pretest scores. American Journal of Physics, 70(12), 1259-1268. Mittlefehldt, S., & Grotzer, T. (2003). Using metacognition to facilitate the transfer of causal models in learning density and pressure. National Association of Research in Using the 5E learning cycle with metacognitive technique to enhance students’ … Runisah, Herman, & Dahlan 98 P-ISSN: 2549-4996 | E-ISSN: 2548-5806 Science Teaching (NARST) Conference, Philadelphia. Retrieved from ftp://202.83.110.129/Volume_1/from%202TB/FAC%20FOUND%20STUDIES/F YLHR9~3/Subject%20Components/Grey%20matters/R54ZK1~G/Project%20Z ero/www.pz.harvard.edu/Research/UCPpapers/P2NARST03.pdf. Mullis, I.V.S., Martin, O., Michael, Foy, P., & Arora, A. (2012). TIMSS 2011 international result in Mathematics. Retrieved from http://timssandpirs.bc.edu/timss2011/downloads/T11_IR_Mathematics_Fullbo ok.pdf Peraturan Menteri Pendidikan Nasional (Permendiknas) Republik Indonesia No. 22 Tahun 2006 tentang Standar Isi untuk Satuan Pendidikan Dasar dan Menengah. (2006). Jakarta: Departemen Pendidikan Nasional. Peraturan Menteri Pendidikan dan Kebudayaan (Permendikbud) Republik Indonesia Nomor 64 Tahun 2013 tentang Standar Isi Pendidikan Dasar dan Menengah. (2013). Jakarta: Departemen Pendidikan dan Kebudayaan. Noer, S.H. (2010). Peningkatan kemampuan berpikir kritis, kreatif, dan reflektif (K2R) matematis siswa SMP melalui pembelajaran berbasis masalah. Dissertation. Bandung: Sekolah Pascasarjana Universitas Pendidikan Indonesia. Ozcan, Z.C., & Erktin, E. (2015) Enhancing mathematics achievement of elementary school students through homework assignments enriched with metacognitive questions. Eurasia Journal of Mathematics, Science & Technology Education, 11(6), 1415-1427. Panaoura, A., & Philippou, G. (2005). The measurement of young pupils ́ metacognitive ability in mathematics: the case of self-representation and self-evaluation. Proceeding of the Fourth Congress of the European Society for Research in Mathematics Education, Spain, 255-264. Rohaeti, E.E. (2010). Critical and creative mathematical thinking of junior high school student. Educationist Journal, 4(2), 99-106. Runisah. (2015). A Study on critical thinking skills in mathematics on eight grade students. Proceeding of International Seminar on Mathematics, Science, and Computer Science Education (pp. 5-10), Bandung. Ruseffendi, E.T. (2005). Dasar-Dasar Ilmu Pendidikan & Bidang Non Esakta Lainnya. Bandung: Tarsito. Slavin, E.R. (2011). Educational psychology: theory and practice, 9th ed. New Jersey: Pearson Education, Inc. Sofuroh, Masrukan, & Kartono. (2014). Model learning cycle 5E dengan pendekatan scientific untuk meningkatkan disposisi matematis dan berpikir kritis. Journal of Mathematics Education Research, 3(2), 91-97. Toit, S.D., & Kotze, G. (2009). Metacognitive strategies in the teaching and learning of Mathematics. Pythagoras, 70, 57-67. Tuna, A., & Kacar, A. (2013). The effect of the 5E learning cycle model in teaching trigonometry on students’ academic achievement and the permanence of their knowledge. International Journal on New Trends in Education and Their Implications, 4(1), 3-87. Yumiati. (2015). Meningkatkan kemampuan berpikir aljabar, berpikir kritis matematis, dan self-regulated learning siswa SMP melalui pembelajaran CORE (Connecting, Organizing, Reflecting, Extending). Dissertation. Bandung: Sekolah Pascasarjana Universitas Pendidikan Indonesia. IJEME, Vol. 1, No. 1, March 2017, 87-98
