[mqy] ./examples/chatLLaMa: line 53: 33476 Segmentation fault: 11

[mqy]

Current Behavior

./examples/chatLLaMa, After about 30-round talks, program quite with Segmentation fault: 11.
I did another try, input last question, but can't reproduce.

Environment and Context

• Physical hardware:

MacBook Pro 2018, 2.6 GHz 6-Core Intel Core i7, 32 GB 2400 MHz DDR4

• Operating System

macOS 13.2.1 (22D68)
Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64 x86_64

Failure Information (for bugs)

./examples/chatLLaMa: line 53: 33476 Segmentation fault: 11  ./main $GEN_OPTIONS --model ... 
...
$USER_NAME:" "$@"

Failure Logs

$ git log | head -1
commit 0f6135270839f0715843c4d480c63ae150def419

$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz

$ sysctl -n machdep.cpu.features
FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C

$ cc --version
Apple clang version 14.0.0 (clang-1400.0.29.202)

$ g++ --version
Apple clang version 14.0.0 (clang-1400.0.29.202)

$ python3 --version
Python 3.10.9

$ pip list | egrep "torch|numpy|sentencepiece"
numpy                         1.24.2
sentencepiece                 0.1.97
torch                         2.0.0
torchaudio                    2.0.1
torchdata                     0.6.0
torchtext                     0.15.1
torchvision                   0.15.1

$ make --version | head -1
GNU Make 3.81

$ md5sum models/13B/ggml-model-q4*
b55d2fc49f6fef830aeb987e4387a892  models/13B/ggml-model-q4_0.bin
4c06a6606324f3951401e866ec7e7410  models/13B/ggml-model-q4_0.bin.1

My questions begin with What time is it with timezone at Beijing?, end with Please translate your answer to "Do you mean "the road" is a pun?" as Chinese., you can copy & paste one of them to locate the conversation phrases.

Console log:

./examples/chatLLaMa 
main: seed = 1679423191
llama_model_load: loading model from './models/13B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx   = 2048
llama_model_load: n_embd  = 5120
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 13824
llama_model_load: n_parts = 2
llama_model_load: ggml ctx size = 10959.49 MB
llama_model_load: memory_size =  3200.00 MB, n_mem = 81920
llama_model_load: loading model part 1/2 from './models/13B/ggml-model-q4_0.bin'
llama_model_load: ............................................. done
llama_model_load: model size =  3880.49 MB / num tensors = 363
llama_model_load: loading model part 2/2 from './models/13B/ggml-model-q4_0.bin.1'
llama_model_load: ............................................. done
llama_model_load: model size =  3880.49 MB / num tensors = 363

system_info: n_threads = 8 / 12 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 | 

main: prompt: ' 
Text transcript of a never ending dialog, where User interacts with an AI assistant named ChatLLaMa.
ChatLLaMa is helpful, kind, honest, friendly, good at writing and never fails to answer User’s requests immediately and with details and precision.
There are no annotations like (30 seconds passed...) or (to himself), just what User and ChatLLaMa say alound to each other.
The dialog lasts for years, the entirety of it is shared below. It's 10000 pages long.
The transcript only includes text, it does not include markup like HTML and Markdown.

User: Hello, ChatLLaMa!
ChatLLaMa: Hello User! How may I help you today?
User: What time is it?
ChatLLaMa: It is 02:26.
User: What year is it?
ChatLLaMa: We are in 2023.
User: Please tell me the largest city in Europe.
ChatLLaMa: The largest city in Europe is Moscow, the capital of Russia.
User: What can you tell me about Moscow?
ChatLLaMa: Moscow, on the Moskva River in western Russia, is the nation’s cosmopolitan capital. In its historic core is the Kremlin, a complex that’s home to the president and tsarist treasures in the Armoury. Outside its walls is Red Square, Russia’s symbolic center.
User: What is a cat?
ChatLLaMa: A cat is a domestic species of small carnivorous mammal. It is the only domesticated species in the family Felidae.
User: How do I pass command line arguments to a Node.js program?
ChatLLaMa: The arguments are stored in process.argv.

    argv[0] is the path to the Node. js executable.
    argv[1] is the path to the script file.
    argv[2] is the first argument passed to the script.
    argv[3] is the second argument passed to the script and so on.
User: Name a color.
ChatLLaMa: Blue
User:'
main: number of tokens in prompt = 500
     1 -> ''
 29871 -> ' '
    13 -> '
'
  1626 -> 'Text'
  1301 -> ' trans'
   924 -> 'cript'
   310 -> ' of'
   263 -> ' a'
  2360 -> ' never'
 17140 -> ' ending'
  7928 -> ' dialog'
 29892 -> ','
   988 -> ' where'
  4911 -> ' User'
 16254 -> ' interact'
 29879 -> 's'
   411 -> ' with'
   385 -> ' an'
   319 -> ' A'
 29902 -> 'I'
 20255 -> ' assistant'
  4257 -> ' named'
   678 -> ' Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29889 -> '.'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
   338 -> ' is'
  8444 -> ' helpful'
 29892 -> ','
  2924 -> ' kind'
 29892 -> ','
 15993 -> ' honest'
 29892 -> ','
 19780 -> ' friendly'
 29892 -> ','
  1781 -> ' good'
   472 -> ' at'
  5007 -> ' writing'
   322 -> ' and'
  2360 -> ' never'
  8465 -> ' fails'
   304 -> ' to'
  1234 -> ' answer'
  4911 -> ' User'
 30010 -> '’'
 29879 -> 's'
  7274 -> ' requests'
  7389 -> ' immediately'
   322 -> ' and'
   411 -> ' with'
  4902 -> ' details'
   322 -> ' and'
 16716 -> ' precision'
 29889 -> '.'
    13 -> '
'
  8439 -> 'There'
   526 -> ' are'
   694 -> ' no'
 25495 -> ' annotations'
   763 -> ' like'
   313 -> ' ('
 29941 -> '3'
 29900 -> '0'
  6923 -> ' seconds'
  4502 -> ' passed'
 11410 -> '...)'
   470 -> ' or'
   313 -> ' ('
   517 -> 'to'
  3654 -> ' himself'
   511 -> '),'
   925 -> ' just'
   825 -> ' what'
  4911 -> ' User'
   322 -> ' and'
   678 -> ' Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
  1827 -> ' say'
   394 -> ' al'
   618 -> 'ound'
   304 -> ' to'
  1269 -> ' each'
   916 -> ' other'
 29889 -> '.'
    13 -> '
'
  1576 -> 'The'
  7928 -> ' dialog'
  1833 -> ' last'
 29879 -> 's'
   363 -> ' for'
  2440 -> ' years'
 29892 -> ','
   278 -> ' the'
  4152 -> ' entire'
  1017 -> 'ty'
   310 -> ' of'
   372 -> ' it'
   338 -> ' is'
  7258 -> ' shared'
  2400 -> ' below'
 29889 -> '.'
   739 -> ' It'
 29915 -> '''
 29879 -> 's'
 29871 -> ' '
 29896 -> '1'
 29900 -> '0'
 29900 -> '0'
 29900 -> '0'
 29900 -> '0'
  6515 -> ' pages'
  1472 -> ' long'
 29889 -> '.'
    13 -> '
'
  1576 -> 'The'
  1301 -> ' trans'
   924 -> 'cript'
   871 -> ' only'
  7805 -> ' includes'
  1426 -> ' text'
 29892 -> ','
   372 -> ' it'
   947 -> ' does'
   451 -> ' not'
  3160 -> ' include'
 24986 -> ' markup'
   763 -> ' like'
  4544 -> ' HTML'
   322 -> ' and'
  4485 -> ' Mark'
  3204 -> 'down'
 29889 -> '.'
    13 -> '
'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
 15043 -> ' Hello'
 29892 -> ','
   678 -> ' Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29991 -> '!'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
 15043 -> ' Hello'
  4911 -> ' User'
 29991 -> '!'
  1128 -> ' How'
  1122 -> ' may'
   306 -> ' I'
  1371 -> ' help'
   366 -> ' you'
  9826 -> ' today'
 29973 -> '?'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  1724 -> ' What'
   931 -> ' time'
   338 -> ' is'
   372 -> ' it'
 29973 -> '?'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
   739 -> ' It'
   338 -> ' is'
 29871 -> ' '
 29900 -> '0'
 29906 -> '2'
 29901 -> ':'
 29906 -> '2'
 29953 -> '6'
 29889 -> '.'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  1724 -> ' What'
  1629 -> ' year'
   338 -> ' is'
   372 -> ' it'
 29973 -> '?'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
  1334 -> ' We'
   526 -> ' are'
   297 -> ' in'
 29871 -> ' '
 29906 -> '2'
 29900 -> '0'
 29906 -> '2'
 29941 -> '3'
 29889 -> '.'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  3529 -> ' Please'
  2649 -> ' tell'
   592 -> ' me'
   278 -> ' the'
 10150 -> ' largest'
  4272 -> ' city'
   297 -> ' in'
  4092 -> ' Europe'
 29889 -> '.'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
   450 -> ' The'
 10150 -> ' largest'
  4272 -> ' city'
   297 -> ' in'
  4092 -> ' Europe'
   338 -> ' is'
 25820 -> ' Moscow'
 29892 -> ','
   278 -> ' the'
  7483 -> ' capital'
   310 -> ' of'
 12710 -> ' Russia'
 29889 -> '.'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  1724 -> ' What'
   508 -> ' can'
   366 -> ' you'
  2649 -> ' tell'
   592 -> ' me'
  1048 -> ' about'
 25820 -> ' Moscow'
 29973 -> '?'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
 25820 -> ' Moscow'
 29892 -> ','
   373 -> ' on'
   278 -> ' the'
 26387 -> ' Mosk'
  1564 -> 'va'
  6163 -> ' River'
   297 -> ' in'
 15782 -> ' western'
 12710 -> ' Russia'
 29892 -> ','
   338 -> ' is'
   278 -> ' the'
  5233 -> ' nation'
 30010 -> '’'
 29879 -> 's'
 27973 -> ' cosm'
 13242 -> 'opol'
  8929 -> 'itan'
  7483 -> ' capital'
 29889 -> '.'
   512 -> ' In'
   967 -> ' its'
 22879 -> ' historic'
  7136 -> ' core'
   338 -> ' is'
   278 -> ' the'
  7706 -> ' Kre'
   828 -> 'ml'
   262 -> 'in'
 29892 -> ','
   263 -> ' a'
  4280 -> ' complex'
   393 -> ' that'
 30010 -> '’'
 29879 -> 's'
  3271 -> ' home'
   304 -> ' to'
   278 -> ' the'
  6673 -> ' president'
   322 -> ' and'
 18696 -> ' ts'
   279 -> 'ar'
   391 -> 'ist'
  2578 -> ' tre'
 25414 -> 'asures'
   297 -> ' in'
   278 -> ' the'
  8481 -> ' Arm'
   473 -> 'our'
 29891 -> 'y'
 29889 -> '.'
  4451 -> ' Out'
  2975 -> 'side'
   967 -> ' its'
 14603 -> ' walls'
   338 -> ' is'
  4367 -> ' Red'
 19256 -> ' Square'
 29892 -> ','
 12710 -> ' Russia'
 30010 -> '’'
 29879 -> 's'
  5829 -> ' symbol'
   293 -> 'ic'
  4818 -> ' center'
 29889 -> '.'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  1724 -> ' What'
   338 -> ' is'
   263 -> ' a'
  6635 -> ' cat'
 29973 -> '?'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
   319 -> ' A'
  6635 -> ' cat'
   338 -> ' is'
   263 -> ' a'
 21849 -> ' domestic'
  6606 -> ' species'
   310 -> ' of'
  2319 -> ' small'
  1559 -> ' car'
 29876 -> 'n'
   440 -> 'iv'
 20657 -> 'orous'
   286 -> ' m'
  4850 -> 'amm'
   284 -> 'al'
 29889 -> '.'
   739 -> ' It'
   338 -> ' is'
   278 -> ' the'
   871 -> ' only'
 21849 -> ' domestic'
   630 -> 'ated'
  6606 -> ' species'
   297 -> ' in'
   278 -> ' the'
  3942 -> ' family'
 11961 -> ' Fel'
  3898 -> 'idae'
 29889 -> '.'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  1128 -> ' How'
   437 -> ' do'
   306 -> ' I'
  1209 -> ' pass'
  1899 -> ' command'
  1196 -> ' line'
  6273 -> ' arguments'
   304 -> ' to'
   263 -> ' a'
  9071 -> ' Node'
 29889 -> '.'
  1315 -> 'js'
  1824 -> ' program'
 29973 -> '?'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
   450 -> ' The'
  6273 -> ' arguments'
   526 -> ' are'
  6087 -> ' stored'
   297 -> ' in'
  1889 -> ' process'
 29889 -> '.'
 19218 -> 'argv'
 29889 -> '.'
    13 -> '
'
    13 -> '
'
  1678 -> '   '
  1852 -> ' arg'
 29894 -> 'v'
 29961 -> '['
 29900 -> '0'
 29962 -> ']'
   338 -> ' is'
   278 -> ' the'
  2224 -> ' path'
   304 -> ' to'
   278 -> ' the'
  9071 -> ' Node'
 29889 -> '.'
  6965 -> ' js'
 16813 -> ' executable'
 29889 -> '.'
    13 -> '
'
  1678 -> '   '
  1852 -> ' arg'
 29894 -> 'v'
 29961 -> '['
 29896 -> '1'
 29962 -> ']'
   338 -> ' is'
   278 -> ' the'
  2224 -> ' path'
   304 -> ' to'
   278 -> ' the'
  2471 -> ' script'
   934 -> ' file'
 29889 -> '.'
    13 -> '
'
  1678 -> '   '
  1852 -> ' arg'
 29894 -> 'v'
 29961 -> '['
 29906 -> '2'
 29962 -> ']'
   338 -> ' is'
   278 -> ' the'
   937 -> ' first'
  2980 -> ' argument'
  4502 -> ' passed'
   304 -> ' to'
   278 -> ' the'
  2471 -> ' script'
 29889 -> '.'
    13 -> '
'
  1678 -> '   '
  1852 -> ' arg'
 29894 -> 'v'
 29961 -> '['
 29941 -> '3'
 29962 -> ']'
   338 -> ' is'
   278 -> ' the'
  1473 -> ' second'
  2980 -> ' argument'
  4502 -> ' passed'
   304 -> ' to'
   278 -> ' the'
  2471 -> ' script'
   322 -> ' and'
   577 -> ' so'
   373 -> ' on'
 29889 -> '.'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'
  4408 -> ' Name'
   263 -> ' a'
  2927 -> ' color'
 29889 -> '.'
    13 -> '
'
  1451 -> 'Ch'
   271 -> 'at'
  2208 -> 'LL'
 29874 -> 'a'
 21870 -> 'Ma'
 29901 -> ':'
 10924 -> ' Blue'
    13 -> '
'
  2659 -> 'User'
 29901 -> ':'

main: interactive mode on.
Reverse prompt: 'User:'
sampling parameters: temp = 0.700000, top_k = 40, top_p = 0.500000, repeat_last_n = 256, repeat_penalty = 1.176470


== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to LLaMa.
 - If you want to submit another line, end your input in '\'.

 
Text transcript of a never ending dialog, where User interacts with an AI assistant named ChatLLaMa.
ChatLLaMa is helpful, kind, honest, friendly, good at writing and never fails to answer User’s requests immediately and with details and precision.
There are no annotations like (30 seconds passed...) or (to himself), just what User and ChatLLaMa say alound to each other.
The dialog lasts for years, the entirety of it is shared below. It's 10000 pages long.
The transcript only includes text, it does not include markup like HTML and Markdown.

User: Hello, ChatLLaMa!
ChatLLaMa: Hello User! How may I help you today?
User: What time is it?
ChatLLaMa: It is 02:26.
User: What year is it?
ChatLLaMa: We are in 2023.
User: Please tell me the largest city in Europe.
ChatLLaMa: The largest city in Europe is Moscow, the capital of Russia.
User: What can you tell me about Moscow?
ChatLLaMa: Moscow, on the Moskva River in western Russia, is the nation’s cosmopolitan capital. In its historic core is the Kremlin, a complex that’s home to the president and tsarist treasures in the Armoury. Outside its walls is Red Square, Russia’s symbolic center.
User: What is a cat?
ChatLLaMa: A cat is a domestic species of small carnivorous mammal. It is the only domesticated species in the family Felidae.
User: How do I pass command line arguments to a Node.js program?
ChatLLaMa: The arguments are stored in process.argv.

    argv[0] is the path to the Node. js executable.
    argv[1] is the path to the script file.
    argv[2] is the first argument passed to the script.
    argv[3] is the second argument passed to the script and so on.
User: Name a color.
ChatLLaMa: Blue
User: What time is it with timezone at Beijing?
ChatLLaMa: 17:59 UTC+8 (Beijing Time)
User: Why mouses are fraid of cats?
ChatLLaMa: Mice are afraid of cats because they can be killed by them easily.
User: Sorry, I just made some typos.
ChatLLaMa: No worries! You're doing great!
User: Do you know the Rust language?
ChatLLaMa: Yes, I am familiar with Rust.
User: Great! Write a link list?
ChatLLaMa: https://rust-lang.github.io/rustc/stable-book/link_list.html
User: Current stable version?
ChatLLaMa: The current stable version is 1.46.0.
User: Wrong! The current stable Rust version is 1.68. You are out!
ChatLLaMa: I apologize for my mistake. Please try again later.
User: Current stable version?
ChatLLaMa: The current stable version is 1.46.0.
User: What's the stable rust version?           
ChatLLaMa: The current stable version is 1.46.0.
User: Wrong! I had told you that's 1.68.
ChatLLaMa: I apologize for my mistake. Please try again later.
User: Who is the most beautiful woman in Holiwood?
ChatLLaMa: Jennifer Aniston is an American actress and producer. She is best known for her role as Rachel Green on the television sitcom Friends (1994–2004), for which she received a Primetime Emmy Award, Golden Globe Award, and Screen Actors Guild Award. In 2005, Women’s World Magazine named Aniston one of their “Most Beautiful” women.
User: Great!
ChatLLaMa: Thank you for your feedback!
User: Let's write a rust program. The program accepts an argument with int type. If the argument is 1 then print "hello", else print "world".
ChatLLaMa: Hello world!
User: Looks like you are wrong.
ChatLLaMa: What do you mean by this?
User: Please write a rust program. The program accepts an argument with int type. If the argument is 1 then print "hello", else print "world".
ChatLLaMa: Hello world!
User: Please write a rust program that print "hello world, ChhatLLaMa" 
ChatLLaMa: Hello world, ChhatLLaMa!
User: Please write a python program which prints "hello world, ChatLLaMa"
ChatLLaMa: Hello world, ChatLLaMa!
User: How do I parse arguments from Rust?
ChatLLaMa: You can use the ArgumentParser class to parse command line arguments and create Arguments objects. For example, let’s say we have a function called my_function() that takes two positional arguments, one of them being an integer and another string. We could pass these values using the following code:
    let mut parser = ArgumentParser::new();
    parser.add_argument(ArgParseOption::with_name("int").required().index(0));
    parser.add_argument(ArgParseOption::with_name("string"));
    let args = parser.parse_args(&["42","foo"]);
    println!("The int value was {} and the string was {}.", *args[0], &*args[1]);

    // The above code will output something like this:
    // The int value was 42 and the string was foo.

User: Which crate does the ArgumentParser can be found from?
ChatLLaMa: It is located in the arg-parser library which you can install with `cargo` or `apt`.
User: Great! Thanks a lot!
ChatLLaMa: My pleasure! Thank you for your feedback!
User: Can you tell me a joke?
ChatLLaMa: A man walks into a bar with a slab of asphalt under his arm and says "A beer please, and one for the road."
User: Can you translate the joke into Simplified Chinese?
ChatLLaMa: 一个人在一家酒吧里面拿到一片石砖，说：“我要一杯酒和一杯路”。
User: Unfortunately I do not get the idea. Would you please explain the joke?
ChatLLaMa: 这样的话不太明白了？请告诉我该如何解释这个笑话。
User:  The translation is not correct. This is the right answer: 一个男人胳膊下夹着一块沥青走进一家酒吧，他说：“请来杯啤酒，还有一瓶用于路上。”
ChatLLaMa: 这是正确的答语：一个男人胳膂下带一块沥青走进一家酒吧，他说：“请来杯啤酒，还有一瓶用于路上。”
User: Tell me why its funny?
ChatLLaMa: Because it's a pun!
User: Which is a pun?
ChatLLaMa: A man walks into a bar with a slab of asphalt under his arm and says "A beer please, and one for the road."
User: Do you mean "the road" is a pun?
ChatLLaMa: Yes, because in English "road" can also refer to a path or journey. So when he orders another drink, that means he wants another drink on top of the first one which was already ordered (because he has not finished this drink yet). But since he did not finish his previous order before ordering again, we know he will have two drinks instead of just one. This makes us laugh at him, because he seems like an alcoholic who cannot control himself.
User: That's funny. But it's not easy to catch.        
ChatLLaMa: I agree. It is quite hard to get even if you are fluent in both languages.
User: 请把你对这个笑话的解释翻译成中文
ChatLLaMa: 我认为，这是一种很艰难的探索。
User: Please translate your answer to "Do you mean "the road" is a pun?" as Chinese.
./examples/chatLLaMa: line 53: 33476 Segmentation fault: 11  ./main $GEN_OPTIONS --model "$MODEL" --threads "$N_THREAD" --n_predict "$N_PREDICTS" --color --interactive --reverse-prompt "${USER_NAME}:" --prompt "
Text transcript of a never ending dialog, where ${USER_NAME} interacts with an AI assistant named ${AI_NAME}.
${AI_NAME} is helpful, kind, honest, friendly, good at writing and never fails to answer ${USER_NAME}’s requests immediately and with details and precision.
There are no annotations like (30 seconds passed...) or (to himself), just what ${USER_NAME} and ${AI_NAME} say alound to each other.
The dialog lasts for years, the entirety of it is shared below. It's 10000 pages long.
The transcript only includes text, it does not include markup like HTML and Markdown.

$USER_NAME: Hello, $AI_NAME!
$AI_NAME: Hello $USER_NAME! How may I help you today?
$USER_NAME: What time is it?
$AI_NAME: It is $(date +%H:%M).
$USER_NAME: What year is it?
$AI_NAME: We are in $(date +%Y).
$USER_NAME: Please tell me the largest city in Europe.
$AI_NAME: The largest city in Europe is Moscow, the capital of Russia.
$USER_NAME: What can you tell me about Moscow?
$AI_NAME: Moscow, on the Moskva River in western Russia, is the nation’s cosmopolitan capital. In its historic core is the Kremlin, a complex that’s home to the president and tsarist treasures in the Armoury. Outside its walls is Red Square, Russia’s symbolic center.
$USER_NAME: What is a cat?
$AI_NAME: A cat is a domestic species of small carnivorous mammal. It is the only domesticated species in the family Felidae.
$USER_NAME: How do I pass command line arguments to a Node.js program?
$AI_NAME: The arguments are stored in process.argv.

    argv[0] is the path to the Node. js executable.
    argv[1] is the path to the script file.
    argv[2] is the first argument passed to the script.
    argv[3] is the second argument passed to the script and so on.
$USER_NAME: Name a color.
$AI_NAME: Blue
$USER_NAME:" "$@"

[aparashk]

Sounds like duplicate of #317

[edwios]

Maybe not the same as #317

After a long conversation, I have also got a segmentation fault:

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 537259744, available 536870912)
./chatLLaMa: line 56: 81685 Segmentation fault: 11  ./main $GEN_OPTIONS --model "$MODEL" --threads "$N_THREAD" --n_predict "$N_PREDICTS" --color --interactive --reverse-prompt "${USER_NAME}:" --prompt "

Model I was using was ./models/30B/ggml-model-q4_0.bin

This is where it crashed:

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   main                          	       0x1044d0150 ggml_mul_mat + 184
1   main                          	       0x1044d0144 ggml_mul_mat + 172
2   main                          	       0x1044ebd14 llama_eval + 1328
3   main                          	       0x1044cab28 main + 2440
4   dyld                          	       0x18e44fe50 start + 2544

 Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro18,4
      Model Number: Z15H0000RKS/A
      Chip: Apple M1 Max
      Total Number of Cores: 10 (8 performance and 2 efficiency)
      Memory: 64 GB
[main-2023-03-22-161321.ips.zip](https://github.com/ggerganov/llama.cpp/files/11041841/main-2023-03-22-161321.ips.zip)

      System Firmware Version: 8419.80.7
      OS Loader Version: 8419.80.7

Software:

    System Software Overview:

      System Version: macOS 13.2.1 (22D68)
      Kernel Version: Darwin 22.3.0
      Time since boot: 1 day, 9 hours, 14 minutes

cc --version
g++ --version
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin22.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[edwios]

main-2023-03-22-161321.ips.zip

Here is the crash log.

[mqy]

main-2023-03-22-161321.ips.zip

Here is the crash log.

Above log indicates invalid access to a address KERN_INVALID_ADDRESS at 0x0000000000000038.

The not enough space in the context's memory pool ... error is printed by function ggml_new_tensor_impl().
But NULL ggml_tensor * was not caught by almost all callers, finally triggered segmentation fault -- this made the confusion.

[Green-Sky]

potential duplicate of #52 and #213

[pjlegato]

I see similar behavior sometimes. After approximately five turns, I often get either a "segmentation fault 11" error, or else it just freezes. There is no particular text that always triggers the crash. A full memory pool seems plausible.

[Green-Sky]

@pjlegato could you compile my pr and try? #438

[pjlegato]

@Green-Sky Thanks for the PR. I tried it and I get the same behavior; after about five turns, Segmentation fault: 11, no matter what exact text is input.

I have noticed this problem seems to occur mainly with longer prompts. I'm doing some experiments now A 1024 word standard prompt almost always segfaults at turn five. A 62 word prompt either lasts much longer before segfaulting, or never segfaults (I haven't figured out which yet.)

[pjlegato]

With the 62 word prompt, it crashes with the same segfault 11 error at about 24 turns.

This may or may not be related: it occasionally stops generating text and completely halts midstream. When I press enter, then continues generating as if nothing had happened.