SuperMemo user survey

SuperMemo World, November 1994

New article by Dr. Piotr Wozniak: The true history of spaced repetition

Here is the statistical analysis of user responses provided in a poll conducted among registered users of SuperMemo in Poland (data gathered in Fall 1994).

The main conclusions coming from the poll are:

  • users of SuperMemo form a very special group of people. A typical user is a highly educated young male non-smoker who exercises a lot and whose main hobby is his computer
  • users of SuperMemo like the program and use it very systematically; they consider SuperMemo one of their most important applications: SuperMemo is the most often used application giving way only to all word-processor applications combined
  1. Education
  2. Profession
  3. Age
  4. Hobby
  5. Health, exercise, sleep and smoking
  6. Version of SuperMemo
  7. Using SuperMemo
  8. Speed of learning(!!!)
  9. Opinions about SuperMemo
  10. Advanced English
  11. Where from did you learn about SuperMemo first time?
  12. What made you buy SuperMemo?
  13. Your greatest achievement
  14. Likes in SuperMemo
  15. Dislikes in SuperMemo
  16. Options used
  17. Options not used
  18. Improvements
  19. Most requested learning material
  20. Fighting piracy
  21. CD-ROM features
  22. Software used
  23. What would make you buy SuperMemo earlier?
  24. What would make your colleagues buy SuperMemo?
  25. Where would you like to find advertising of new SuperMemo products (in Poland)?
  26. Other: Where are women?

Education (of users taking part in the poll)

  • 3% Ph.D.
  • 45% MSc
  • 27% university students
  • 11% secondary education
  • 14% secondary school students
  • 1% primary school students
  • 0% primary education

Profession

  • 26% engineer (of which about half are computer scientists)
  • 25% university student (of which 33% computer science, 20% sciences, 16% electronics, 15% economics, 7% medicine, etc.)
  • 15% secondary school student
  • 8% businessman (usu. putting himself or herself in the engineer category)
  • 6% scientist
  • 5% teacher
  • 5% technician
  • 3% pensioner
  • 2% lawyer
  • 9% other professions (e.g. military, librarian, policeman, priest, etc.)

Age

The average user is 29 years old. The youngest respondent was 16 years old, the oldest 54. 14% of users are above 40, 27% are less than 20.

Hobby

  • 64% computers
  • 20% sport
  • 18% foreign languages
  • 9% music
  • 8% tourism
  • 8% math
  • 6% photography
  • 5% each of the following: history, philosophy, chemistry, science fiction, politics, business
  • 4% each of the following: books, psychology, physics

Other frequent responses: movies, social sciences, pedagogy, logic, biology, electronics, architecture, astronomy, education, garden, do-it-yourself, Bible, cars, etc.

Smoking, sleep and exercise Only 4% of users of SuperMemo smoke! For comparison, 30% of Americans and 70% of Japanese are smokers (1990)! An average user sleeps 7.5 hours per day. Smokers sleep 1.1 hours less. Those who sleep more than 7.5 hours had memorized 3870 items on average, while those who slept less than 7.5 hours had memorized 4570 items. An average user exercises 27 minutes per day.

Version of SuperMemo

  • 53% of respondents reported using SuperMemo 7 for Windows (an increase from 11% a year ago). Only
  • 8% reported using versions newer than SuperMemo 7.2.
  • 14% used SuperMemo shareware before buying commercial SuperMemo (108 days on average).
  • 12% admitted having used an illegal copy of SuperMemo beforehand (98 days on average). In 1993, the same figure was 6%.

Among learners who obtained both Polish and English version of SuperMemo, 52% used the English version, 48% used Polish version.

Using SuperMemo

An average learner has been using SuperMemo for 11 months, 28 minutes per day, and has memorized 4670 items (equivalent of 182 items/year/minute, i.e. substantially less than the average acquisition rate; see Speed of learning). This data is almost identical with the one obtained a year earlier (10 months, 27 minutes, 4400 items).
87% of users reported breaks in learning (note, however, that the break-handling option Mercy belonged to the group of least used options in SuperMemo).
The longest break was on average 19 days long (excluding innumerous cases, about 1%, in which users reported breaks longer than 6 months).
The average proportion of breaks in learning in the learning process was reported to be 9%.

Speed of learning

The average speed of learning was 243 items/year/minute. The same value after rejecting scores less than 100 and more than 600 was 242 items/year/minute. Those who sleep less than 7.5 hours learn at the speed of 240 items/year/minute. Those who sleep more than 7.5 hours learn at 256 items/year/minute. Apart from the above finding, factors such as smoking, sleep and exercise are poorly correlated with the speed of learning.
Learners 28 years old or younger learn at the speed 264 items/year/minute, while those above 28 years learn at the speed 179 items/year/minute. Remember, however, that in another study it has been shown that good learners learn slower (!), because of their greater self-criticism in providing grades.
At the reported average speed of learning, a 1 hour per day learning sessions make it possible to memorize Basic English (3000 items) in 2.3 months, and Advanced English (40,000 items) in 2.5 years (note, however, that the speed of learning may be much higher in the first few months of working with a given database).
80% of users confirmed that it is possible to learn Basic English in just 30 days working 0.5 hour per day.
78% believe that it is possible to memorize Advanced English in four years working 0.5 hour per day.

The best reported result of memorizing Advanced English comes from Marek Minakowski (Institute of Philosophy, Jagiellonian University, Krakow), who memorized 12,000 items in just four months. Second best was Jaroslaw Ast, (student of dentistry, Medical Academy of Poznan), who memorized 10,000 items in four months. Pawel Dzierwa (Warsaw) who held the best result a year earlier (10,000 items) started his work all over again upon receiving CD-ROM with Audio Advanced English.
Piotr Wozniak (SuperMemo World, the author of the database), memorized the entire Advanced English, but he passed only 10,000 items in his first year. We are waiting for similar reports from other learners.

Opinions about SuperMemo as a method and as software

The users have the following opinions about the SuperMemo method:

  • 69% an excellent aid in learning
  • 30% useful idea
  • 1.5% nothing special
  • 0% artificial approach to learning
  • 0% waste of time

On the grade point scale from 0-worst to 5-best, the method scored 4.67 on average (4.75 a year earlier). Surprisingly, the grade was poorly correlated with the reported speed of learning and with the number of memorized items. It was also poorly correlated with the version of the program and with the student's education.

The following are users’ opinions about SuperMemo as a program:

  • 16% excellent piece of software
  • 63% reliable product
  • 20% useful product that must be improved
  • 0% product full of bugs
  • 0% trash

On the grade point scale from 0-worst to 5-best, the program scored 4.0 on average (4.17 a year earlier). Again surprisingly, the grade was poorly correlated with the version of the program. Additionally, learners who obtained higher learning speeds were slightly more critical about software, while older users tended to provide slightly higher grades. Education had no bearing on the grade.

Opinions about the Advanced English database

  • 40% excellent aid in learning English
  • 15% useful database that needs improvement
  • 9% useful database that contains awkward items and definitions
  • 0% low quality product
  • 36% not used (or no response)

Where from did you learn about SuperMemo first time?

  • 88% press (of which: 50% advertising, 25% press reviews and 25% leaflet insert)
    • 26% PC Kurier
    • 8% Bajtek
    • 6% Enter
    • 5% Computer World
    • 2% Informatyka
  • 14% friend
  • 6% leaflet
  • 5% shop
  • 2% software fair
  • 1% computer market

What made you buy SuperMemo?

  • 58% reviews in press
  • 55% scientific method employed in SuperMemo
  • 53% possibility to create my own databases
  • 39% belief in the effectiveness of SuperMemo
  • 34% awards given to SuperMemo
  • 26% leaflets
  • 23% recommendation of a colleague, shareware, illegal version, demo (10%, 6%, 5%, 2%)
  • 23% one or many adverts (11% one, 11% many)
  • 19% availability of many databases
  • 18% multimedia capability
  • 15% availability of a Windows version
  • 8% software fairs
  • 3% shopkeeper's advice
  • 3% nice box
  • 3% poster
  • 3% buying by accident
  • 0% company's purchase
  • 0% recommendation by a superior

What is your greatest achievement with SuperMemo?

  • 26% memorizing a great deal of items/words/phrases
  • 24% no answer
  • 18% becoming regular in my work
  • 9% quickly memorizing Basic English
  • 8% gaining self-confidence (language ability, memory ability, etc.)
  • 8% passing an important exam (in 60% of cases: language exam)
  • 6% learning a foreign language
  • 5% regular growth of foreign language vocabulary
  • 4% no success
  • 2% satisfaction

Other responses: 500 items memorized in a day, 8-th place in the database competition, excellent grades in language courses, best grades in English in my school, qualifying to the finals of Latin Language Contest, memorizing “drumrum” in German (at last), learning several programming languages, starting to like German, memorizing 10,000 items in a year, etc.

What do you like most in SuperMemo?

  • 49% effectiveness
  • 25% the fact that it enforces regular work
  • 11% ability to create one’s own databases
  • 9% simplicity
  • 6% no need to type in the correct answer
  • 5% self-grading
  • 5% possibility to edit ready-made databases
  • 4% database statistics
  • 4% possibility of adding graphics and sound
  • 3% developing the program along users’ suggestions
  • 2% fast development of SuperMemo software
  • 2% application of a scientific method
  • 2% minimum information principle
  • 2% reliability
  • 1% large number of databases
  • 1% the fact that it inspires creativity

Other responses: satisfaction, "everything", regulated speed of learning, etc. The four most liked features are exactly the same as in a survey presented a year earlier!

What do you dislike most in SuperMemo?

  • 22% I like everything
  • 18% dislikes in reference to SuperMemo 6 for DOS
  • 7% the fact that I have to work systematically
  • 5% no possibility to resize question and answer area (available as of SuperMemo 7.3)
  • 5% incomplete Polish manual (esp. unintelligible description of the options on the Analysis menu)
  • 5% no possibility to change a mistakenly given grade (available in SuperMemo 8)
  • 5% International Phonetic Alphabet is not used in Basic English nor Advanced English (available in SuperMemo 8)
  • 4% upgrades are too expensive
  • 4% Database Toolkit is not integrated with the program (integrated as of SuperMemo 7.4)
  • 3% the need to recover unclosed databases (not relevant in SuperMemo 8)
  • 3% self-grading
  • 3% no way to stop repetitions during a vacation period
  • 3% bugs (in reference to version 7.1 and 7.2 for Windows)
  • 3% program is too big (or too slow)
  • 2% long response time from SuperMemo World (test this e-mail to see if this is still true: SuperMemoMail)
  • 2% typing in items when creating my own database
  • 2% the size of the main window is not remembered (available as of SuperMemo 7.3)
  • 2% poor quality help (in reference to the Polish version)
  • 2% no way to upgrade Advanced English upon getting a new release
  • 2% poorly designed interface (in reference to SuperMemo 7.1 and 7.2)
  • 2% bugs in databases
  • 2% too many parameters displayed on the screen (in SuperMemo 7.5 the number of parameters can be restricted by the user)
  • 1% databases are too expensive
  • 1% grades cannot be fractional
  • 1% no way to type in the correct answer (available as of SuperMemo 7.4)
  • 1% searching only one database at a time

Other responses: it is difficult to edit wave files, strange database format, the need to work on weekends, it is too easy to delete items, renumbering items at Garbage, slow Search, too few databases, no spell checker, imposing an English version on Poles, no autostart program that would show the number of outstanding items each time one starts Windows, exaggerating the increase in the speed of learning (50 times?), AdvEng manual in English, deleted items leave empty places in a databases (until Garbage), etc.

Which options do you use most often (apart from Open, Learn and Add new items)?

  • 34% Search
  • 32% Analysis
  • 23% Burden
  • 17% Edit
  • 17% Garbage (not needed in SuperMemo 8)
  • 13% Monthly burden
  • 13% Interval distribution
  • 11% A-factor distribution
  • 8% Random test
  • 6% I use all options
  • 5% Date

Which options do you never use?

  • 23% Wipe (replaced with intelligent Mercy in SuperMemo 8)
  • 16% Mercy
  • 14% Tools : Analysis : Matrices
  • 14% Random test
  • 8% Help
  • 7% Garbage (not needed in SuperMemo 8)
  • 5% Date
  • 4% Approximate (not needed in SuperMemo 8)
  • 3% Search

Proposed improvements to SuperMemo

  • 13% good manual in Polish (the survey was done in Poland only)
  • 12% possibility of changing once-given grade (available in SuperMemo 8)
  • 4% possibility to type in the correct answer (implemented as Spell-Pad in SuperMemo 7.4)
  • 4% integrated database toolkit (implemented in SuperMemo 7.4)
  • 3% printing a range of items (implemented in SuperMemo 7.5)
  • 3% associating images with answers (available in SuperMemo 8)
  • 3% resizing question and answer window area (implemented in SuperMemo 7.3)
  • 3% possibility to merge databases (included in Database Toolkit in SuperMemo 7.4)
  • 3% transferring items between databases (implemented in SuperMemo 7.4)
  • 3% swapping questions with answers (implemented in SuperMemo 7.4)
  • 3% help in Polish as good as in the English version (the survey was done in Poland only)
  • 3% importing DBF files (possible upon converting DBF files to text files)
  • 3% different font for question and answer fields (implemented in SuperMemo 7.4)
  • 3% automatic database back-up with database compression
  • 3% adding sound to Advanced English (available on SuperMemo 7.4 CD-ROM)
  • 2% associating many items with one bitmap file (implemented in SuperMemo 7.4)
  • 2% multiple fonts in the same item (available in SuperMemo 8)
  • 2% keep the size of the main window in INI files (implemented in SuperMemo 7.4)
  • 2% sharing database files between users (available in SuperMemo 8)
  • 2% providing a name of a database to open as a parameter of the program (implemented in the Windows version)
  • 2% searching only question fields (implemented in SuperMemo 7.3)
  • 2% word-wrapping in the item editor (implemented in SuperMemo 7.2)
  • 2% case-insensitive Search (implemented in SuperMemo 7.4)
  • 2% possibility to change the sequence of items in a database (implemented in SuperMemo 7.4)
  • 2% Random test on difficult items (implemented in SuperMemo 7.4)
  • 1% Random test on memorized items (implemented in SuperMemo 7.4)
  • 1% use International Phonetic Alphabet in transcription (available in SuperMemo 8)

Other interesting propositions included: real learning time measurement, context-sensitive help, verbal description of statistical parameters, colorful dialog boxes, nicer icon, keep a list of databases to open in sequence, weekly calendar with no repetitions on selected days, different keyboard layouts associated with databases, more informative list of available databases (DBANK.TXT), search going through a list of databases, integrated on-line dictionary, screen saver, viewing items falling into a selected E-factor category, removing bugs from databases Basic English and Advanced English, insert graphics into the text of items, possibility to sort items alphabetically, spellchecker, compression of graphic files, memo field for user comments, program for activating SuperMemo upon starting Windows in case the number of outstanding items is greater than zero, etc.

Most requested learning material

  • 7% English (e.g. Advanced English in Polish, Intermediate English, Technical English, etc.)
  • 5% German (e.g. Advanced German, Basic German, etc.)
  • 5% History
  • 5% Russian
  • 3% French
  • 2% Geography
  • 2% Physics
  • 2% Math
  • 2% Chemistry
  • 1% Biology

Other neglected fields: Electronics, Telecommunications, Economics, Esperanto, etc.

What should SuperMemo World do to make illegal users legalize their copy?

  • 25% keep low upgrade prices for legal users
  • 17% reduce the price of SuperMemo
  • 15% competitions, reward draws, newsletters, promotions, surprises, SuperMemo Club, etc.
  • 11% one-time legalization action (legalizing on special conditions)
  • 10% let illegal users register at a lower price (see remarks below)
  • 8% one-time amnesty
  • 5% seasonal decrease of the prices (e.g. before Christmas)
  • 5% let them register in return for a database (NB: this has always been an option for illegal users)
  • 4% constantly develop SuperMemo as software
  • 4% release a CD-ROM with all available databases
  • 3% develop more databases
  • 3% let them buy the manual at reasonable price
  • 3% hunt and punish illegal users (see remarks below)
  • 2% release shareware version (NB: SuperMemo 6 is available in public domain)
  • 2% copy-protection (see remarks below)
  • 1% there is no way to make it happen
  • 1% bundle SuperMemo with other products, computers, etc.

Other opinions: "SuperMemo World did all it could be letting illegal users register in return for a good database!", Include biblical verses indicating sinful nature of piracy, etc.

Remarks: SuperMemo World does not consider allowing registration at a lower price as it does not seem fair to those who paid the full price. Similarly, SuperMemo World does not consider punitive actions against illegal users hoping that they can be attracted by affirmative actions. Finally, copy protection cannot be considered because inconvenience it imposes on legal users.

What would you like to get on SuperMemo CD-ROM?

  • 17% not interested (e.g. because of lack of CD-ROM drive)
  • 10% hypermedia collections
  • 10% many collections (not necessarily multimedia)
  • 8% tools for editing collections
  • 4% dictionaries
  • 3% games
  • 2% I do not know
  • 2% Advanced English Audio (available on Advanced English CD-ROM)
  • 1% English Grammar Audio (available on Multilingual CD-ROM)
  • 1% Basic English Audio (available on Advanced English and Multilingual CD-ROMs)
  • 1% German Audio (available on Multilingual CD-ROM)
  • 1% full manual (available in English on all CD-ROMs)

Others: French Audio (available on Multilingual CD-ROM), Russian Audio (available on Multilingual CD-ROM), English Vocabulary (by M.Cajza), demo, Picture Editor, songs in foreign languages, etc.

Which application software do you use most often?

Note that the survey was conducted in 1994 therefore you will not find many of the currently popular applications on the list! Endemic Polish software is marked with PL.

Most often used application software (among questioned users of SuperMemo);

  • 39% word processors, of which:
    • Word 13%
    • TAG 9% (PL)
    • AmiPro 7%
    • Chiwriter 4%
    • WordPerfect 2%
    • other 4%
  • 37% SuperMemo
  • 16% spreadsheets (mainly Excel, Quattro Pro, 1-2-3)
  • 15% Borland Pascal
  • 11% database systems (mainly FoxPro, Access, dBase)
  • 5% dictionaries (mainly Skalmierski’s dictionary)(PL)
  • 5% C, C++
  • 5% DTP and graphics (mainly Corel, Picture Publisher)
  • 4% educational software (most popular: Media Euro (PL), Pop English (PL), Pop Deutsch (PL), E-Teacher (PL), E-Teacher Deutsch (PL))
  • 3% Auto CAD
  • 3% scientific (Derive, TEX)
  • 2% professional applications related to one’s job
  • 1% assembly languages (mainly MASM)
  • 1% games (Doom)
  • 1% other applications (e.g. antiviral software, PKZIP, Works for Windows, Organizer, Makler (PL), Quicken, ARJ, e-mail software, modem software, etc.)

What would make you buy SuperMemo earlier?

  • 50% getting to know it earlier
  • 19% lower price
  • 10% fewer slogans, more facts (e.g. in advertising)
  • 6% getting SuperMemo shareware
  • 2% recommendation of other users
  • 1% getting samples of databases
  • 1% greater availability in shops

What would make your colleagues buy SuperMemo?

  • 34% lower price
  • 14% more concrete information in press
  • 11% more positive opinions from friends
  • 5% more databases in the offer
  • 5% using shareware
  • 4% more motivation for work on their part
  • 3% access to a computer

Where would you like to find advertising of new SuperMemo products (in Poland)?

  • 47% PC Kurier
  • 41% Enter
  • 14% Gazeta Wyborcza
  • 13% Bajtek
  • 11% PC Magazine
  • 9% PC World
  • 8% Chip
  • 6% Computer World
  • 5% foreign press (Byte, PC World, etc.)
  • 5% Wiedza i Zycie
  • 3% Informatyka
  • 3% Rzeczpospolita
  • 3% daily newspapers

Other statistics

Only 3% of respondents were women!!!

Although 60% of users claimed that SuperMemo for DOS should continually be developed, SuperMemo World gave up the development because of a dramatic drop in user inquiries. We decided to release SuperMemo 6 Public Domain for DOS instead.

Optimum price for SuperMemo on floppy disks in Poland was proposed to be PLN 76 (about $20).

36% of respondents were interested in buying a CD-ROM with all available SuperMemo databases

See also: Survey 1999 and Survey 1993

1.3.51