lokki i get 4 dropped blocks, but that is to be expected i guess

yes, expected.

lokki g called presets2.wavpresets, i have no idea where that extra presets is coming from.

that is a C thing, where strings do not actually exists, but are sequences of chars terminated by a null character (\0). The length of the string "presets2.wav" is therefore 12+1 = 13. When you call

   sprintf(fileName, "presets2.wav");
   sprintf(tableName, "presets");

the first string actually exceeds the space allocated in fileName, and effectively writes past the end of the array, causing undefined behaviour. In this case, the behaviour is that fileName and tableName are allocated contiguously, so that the \0 terminating character for the fileName string actually ends up being the first element of the tableName string, and the second line overwrites that. Remember that strings are \0-terminated, which means that when you start reading a string from a given position in memory, the end of the string will be the first \0 character encountered. As a result, if you start reading from tableName, you get presets, but if you start reading from fileName, you will read presets2.wavpresets. Whatever, just increase the size of those char arrays. You should also use snprintf instead, which is safer (check man snprintf). But ultimately, in case you don't actually need to change your strings dynamically, there is no need to do any of that. This will do:

const char fileName[] = "presets2.wav";
const char tableName[] = "presets";

lokki is it a problem to use "4" in the writeSamples() for the frames input?

It is not a problem.

lokki i don't know how to get the size of the table from PD

According to this there is a unsigned int hv_table_getLength(HeavyContextInterface *c, unsigned int tableHash); that should do it.

lokki ok when i change the sfinfo.format = SF_FORMAT_WAV | SF_FORMAT_FLOAT;
to: sfinfo.format = SF_FORMAT_WAV | SF_FORMAT_PCM_16;
the name is correct. however there is no data written in that case, file is 0.0kb and audacity can still not open it.

not sure what is happening there. Can you not go for PCM_16 ? Note 1 at the bottom of the docs is a bit confusing for me as to what float formats do, and my hint would be: stay away from them unless you actually need them (not that this explains why your code stops working when you do that).

              giuliomoro
              thanks as always for your support!! i replaced the name variables with constants and indeed it works now, great! actually that 12 in there was a typo in the first place.

              and, i got bitten by the "heavy or IDE not updating all the files it should" bug... once i deleted the whole heavy destination folder and recompiled everything the data is now correctly written out into the wav file, great! i did not have to change anything in the code...

              thanks again, i will now try to implement it into my actual patch.

                  lokki heavy or IDE not updating all the files it should" bug... once i deleted the whole heavy destination folder

                  This may happen if you have a file open in the IDE and then you change that file from the terminal: the IDE doesn't actually monitor file changes on disk. It assumes it has full control

                    lokki thanks again, i will now try to implement it into my actual patch.

                    and it all works flawless! i was wondering in the process if it is worth the trouble to use heavy, but first of all my CPU dropped from 80 % to 28 % and second the latency is much smaller, and since this is a vocal live processor this was very important! thanks again.

                      8 days later

                      ok, coming back to this...

                      there is a problem with my code or the way it is executed.

                      every now and then the presets.wav file gets garbage data when i write to it which makes the presets useless in the long run.

                      could it be that sometimes the data cannot be written right because it runs in the audio-thread? at this point i am writing 400 samples into a wav file and i get 14 blocks dropped when i do so. 9 out of 10 times it works but if it doesn't the generated wav file has not much to do with my presets anymore. is there a way to write the samples in an auxiliary task to avoid this problem?

                      i am running the system off the internal memory and also save on there, could this be the problem? (i can switch to an sdcard if that is the issue)

                        lokki o. 9 out of 10 times it works but if it doesn't the generated wav file has not much to do with my presets anymore.

                        can you describe what is in the generated "corrupted" wav file? Try and save several copies of the file without changing the preset of the content, so that you would expect it to generate exactly the same file over and over again. Set aside a "good" one, and a "corrupted" one.
                        First off, check if the size is exactly the same in both cases. Then use hexdump to print a hex dump of the files and see where the differences are. You would expect that the header of the two .wav files would be the same, and that there is some difference in the data part, perhaps.

                        lokki could it be that sometimes the data cannot be written right because it runs in the audio-thread?

                        This shouldn't be a problem, unless you suddenly kill the board without shutting down gracefully. If that is the case, when the file is corrupted you'd expect it to be shorter, or empty. You should probably add a call to sync()(see man page ) after sf_close(), to ensure the file is correctly synced to disk and not stored in a buffer by the OS.

                              this gives you an idea of the difference between close() (which implies flush()) and the data being actually written to disk, which should happen normally after a while, or can be forced withsync().

                              giuliomoro This shouldn't be a problem, unless you suddenly kill the board without shutting down gracefully. If that is the case, when the file is corrupted you'd expect it to be shorter, or empty. You should probably add a call to sync()(see man page ) after sf_close(), to ensure the file is correctly synced to disk and not stored in a buffer by the OS.

                              thanks for the hint, will do this! however:

                              the corrupted presets file is exactly the same in size but has just random 1 and even a -1 written to it.

                              alt text

                              (above is the wrong file, below a correct preset file)

                              i had one other suspicion. i call the libsdnfile write process immediately after i write to the table within heavy. i do this with a
                              [t b b] to make sure the call is after the write. however there is quite some stuff going on in the first bang, so maybe the second bang is incorrectly scheduled before everything finishes, and hence data is written to disk while the array changes which is not a very good idea. i changed this now to make sure it starts after the last write into the array. let's see if i get any more false writings...

                                  ok, seems it is not the writing after all. i just got the bad preset loading again. (all sounds are garbled) i stopped the bela process and looked at my presets file, it was still intact. i started again, got the same garbled presets, and now saved a preset. no the file was completely useless as well. so it seems the reading (which happens in setup) sometimes fails and corrupts the data.

                                  and... i found the bug.

                                  i realised that when the presets where garbled this message was printed on the bela console:

                                  File samples scale = 16.2277

                                  which of course made my presets values to large by an amount of 16.227

                                  i now changed the offending line and sure enough it works again :-)

                                  if (subformat == SF_FORMAT_FLOAT || subformat == SF_FORMAT_DOUBLE) {
		double	scale ;
		int 	m ;

		sf_command (sndfile, SFC_CALC_SIGNAL_MAX, &scale, sizeof (scale)) ;
		if (scale < 1e-10)
			scale = 1.0 ;
		else
		//	scale = 32700.0 / scale ;
		scale = 1.0 ;
		cout << "File samples scale = " << scale << endl;

		for (m = 0; m < frameLen; m++)
			tempBuf[m] *= scale;
	}

                                  so it seems that the signal max calculation is sometimes way off in getSamples()

                                    lokki the corrupted presets file is exactly the same in size but has just random 1 and even a -1 written to it.

                                    this suggests it's something wrong on the heavy/C++ side of things then.

                                    lokki i had one other suspicion. i call the libsdnfile write process immediately after i write to the table within heavy. i do this with a
                                    [t b b] to make sure the call is after the write. however there is quite some stuff going on in the first bang, so maybe the second bang is incorrectly scheduled before everything finishes, and hence data is written to disk while the array changes which is not a very good idea.

                                    I don't see why this would be a problem. The execution is single-threaded, so there is no way I can think of that you would be able to get a non-deterministic behaviour because of concurrency. Is it possible that there is some non-deterministic behaviour in the patch? I.e.: are you sure that you are sending the message to write the table to file only when the table is in a valid state?

                                    Another test to try and localise the issue would be to dump a regular C array instead of whatever you get from heavy.

                                          giuliomoro Another test to try and localise the issue would be to dump a regular C array instead of whatever you get from heavy.

                                          see my previous post, the issue was with the getSamples()scaling function...

                                            lokki so it seems that the signal max calculation is sometimes way off in getSamples()

                                            Yes, I removed it in more recent versions of SampleLoader.h: https://github.com/BelaPlatform/Bela/issues/542

                                            I am a bit confused as to why and how, if the issue was in getSamples(), it managed to affect the written file and not the one you read.

                                                giuliomoro I am a bit confused as to why and how, if the issue was in getSamples(), it managed to affect the written file and not the one you read.

                                                yes this was because i saved a preset when i already had the garbled file loaded which of course did also create a garbled preset file on writing (since it wrote back the bad loaded data). at that point i did not know that it was the reading bit and assumed (since i wrote the writing part) that that had to be broken...

                                                  giuliomoro Yes, I removed it in more recent versions of SampleLoader.h

                                                  whoops, that is what you get for not updating regularily. sorry for the noise then...

                                                    or for not following this recommendation:

                                                    giuliomoro float formats do, and my hint would be: stay away from them unless you actually need them

                                                    Funny thing: in that sentence I was referring to libsndfile's pitfalls when using floats, and it turned out we had some on our side as well 🙂