1 (edited by Shiru 2017-12-06 17:56:58)

Topic: In-game (during gameplay) music

We're explored a lot in the area of stand alone beeper music, one that takes nearly all CPU time, but only scratched the surface of the music that plays along some action on the screen. Could be interesting to try this out too. Here is my thoughts.

Early ZX games often were attempting to play some music alongside the gameplay - for example, Antics or Manic Miner.

There is three methods to play beeper music during gameplay.

First is to just put a very short duration beep after each frame. Causes major slowdown to the game and staccato sounding music, but with relatively clean sound.

Second, very often employed to play sound effects, is to play short duration beeps and noises during leftover time of a TV frame, instead of HALT, sound loop stopped with a new interrupt. It introduces major 50 Hz buzz, but no slowdown. Requires to have enough leftover time, though.

Third, I think is the most obscure one, is applicable to games where frame time is evenly distributed around some nearly constant-time process (rendering usually), so they can call sound routine a number of times per frame with mostly constant pace. That likely would suit well for multicolor engines such as Nirvana. They can have a short sample buffer, prepared outside the raster, then read and output value from the buffer to #FE each N scanlines (very low sample rate, but may work for some cases).


The result in existing old games, I think, almost unversally could be described as 'annoying'. But I don't think that's just because of some general issue with the idea itself, rather with poor execution. Most games with in-game music use monophonic single beeps, single simplest timbre (square), combined with poor music (mostly crude renditions of classics), mostly without any pauses (constant beeps/buzz), no dynamics at all. If either or both of the issues (tech and composition) will be improved, the result should become much more pleasant.

i don't think polyphony is a must, as my practice (yet to be released) with PC Speaker shown that it is very well possible to pull quite impressive illusion of multiple voices with proper authoring tools (easy multiple-to-single channel voice management) and composing. Nice to have anyways, if possible.

Timbre/tone control would help greatly, especially pseudo volume control - it would bring the dynamics, and timbre control would help with sound variety. Noise generator would improve the percussion too, even though PC Speaker experience shows that even square tone alone is enough to make a good rythm parts.

Proper composition of new original music, with good variety and dynamics, is of course would bring the greatest improvement to the matter.

website - 1bit music - other music - youtube - bandcamp - patreon - twitter (latest news there)

Re: In-game (during gameplay) music

Very interesting topic Shiru.

The best I've seen is a mini game that FrankT wrote which had full speed special FX music playing throughout.

It was a simple game though (a recreation of an 80s handheld 'ambulance' LCD game). Very impressive.

And ofcourse there's introspec's rain demo.

The 1986 game 'thunderbirds' had  multichannel wham the jukebox music playing through too I seem to remember

Re: In-game (during gameplay) music

It's here:

https://youtu.be/bFJLZWDOA-M

Re: In-game (during gameplay) music

I wonder if a new rombeep player could be tweaked to play on interrupt ?

Btw, an interesting fact I found is that rombeep tunes (compiled on beepola) don't lock up currah speech playback so essentially it can sing over a backing track !

Re: In-game (during gameplay) music

Here is rough outline how the second approach could work. It needs to have IM2 interrupts and HALT before a new game frame. Does not matter if there is few TV frames per game frame, but more constant CPU load time is preferred, to avoid major variations of the sound bursts duration.

Once program needs to perform HALT, it rather calls HALT_SE.

HALT_SE:
restore all registers required for sound/music generation
LOOP:
generate sound, needs to be the usual short loop
scf
BREAK=$
ccf
jp nc,LOOP
restore ccf opcode at BREAK
store back all registers, the point is to be able to smoothly resume sound generation during next call
ret


IM2 interrupt should just replace ccf at BREAK with nop, thus breaking the sound generation loop (could be any other way).

website - 1bit music - other music - youtube - bandcamp - patreon - twitter (latest news there)

Re: In-game (during gameplay) music

I'm mostly interested in the 3rd approach, especially in combination with multi-color gfx. To me this combination totally makes sense because both these things require exact timing. I'm also curious how a perfectly timed engine would sound in the first place. Did some tests with this back in November but didn't manage to complete anything before leaving for Germany.

Re: In-game (during gameplay) music

It may sound crazy, but if the game is simple, you can use a beeper engine that uses IM2 interrupt mode (like Special FX or Plip Plop) and run the game on interrupt.
Be warned that both the sound quality and game speed will be lower. This is why this must be a simple game.

AYGOR and computer projects: http://abrimaal.pro-e.pl/zx/
Abrimaal music: http://abrimaal.pro-e.pl/music/

Re: In-game (during gameplay) music

It will work, but basically it just inverts the idea of playing sound in spare CPU time that is left of a frame - the second apporoach I described above. Just different technical implementation.

Engines like Special FX work better with this approach, because of very short pins they're generating.

website - 1bit music - other music - youtube - bandcamp - patreon - twitter (latest news there)

9 (edited by cmgonzalez 2019-03-20 14:49:16)

Re: In-game (during gameplay) music

Shiru wrote:

Third, I think is the most obscure one, is applicable to games where frame time is evenly distributed around some nearly constant-time process (rendering usually), so they can call sound routine a number of times per frame with mostly constant pace. That likely would suit well for multicolor engines such as Nirvana. They can have a short sample buffer, prepared outside the raster, then read and output value from the buffer to #FE each N scanlines (very low sample rate, but may work for some cases).

Hi Shiru, first to all, I want to thank you for your mFX player that I'm already using for my games Gandalf and Manic Pietro. Currently I'm working on a Pacman clone and I want to target the 48, with beeper sound fx. Do you have any code example for that obscure solution that you mention?


I've run some test, this are my findings (I'm a newbie on the 48 beeper)

-Sounds can be placed as long they don't mess with interruptions, for example the Basic Beep routine can be used on short intervals and also out254 based sounds seems to be work ok, with no screen flickering or corruption.

-Also I've try to use BeepFx but it disables interrupts. sad

-Steve Turner player works but it may be interfering with some registers that SDCC needs, as the code jumps to an unexpected place.

-I've placed a call to tiny out254 routine on the Nirvana ISR, to make some noise each time pacman eat a dot and works ok.

What do you think, Any idea or suggestions?

Regards

10 (edited by utz 2019-03-24 12:31:19)

Re: In-game (during gameplay) music

Hi,

Since Shiru seems to be busy, I'll try to answer in his place.

I don't think there is any code for the method Shiru describes, because nobody has implemented it. The idea for continuous sound would be not to call the sound generator through an ISR (because 50 Hz is too slow to make anything but the lowest audible tones), but to interleave it with the multicolour code at regular intervals. The usual restrictions with contention apply there, so it would still be quite tricky. Generally, PFM/pin pulse and Zilogat0r's Squeeker method would work best because they both can work with a low sample rate.

For sfx and simple melodies consisting of short blips, doing this on interrupts works fine. Avoid the Basic Beep, as it has a lot of overhead. You can achieve the same result with just three commands:

  add hl,de
  ld a,h
  out (#fe),a

Init DE with the desired frequency divider (for tests, something like #0040 will be fine), and run in a loop for something like 1000 times.

In z88dk, there's also a modified version of the Tritone engine with can be used in combination with game logic. It basically runs all the time and then periodically runs your game code on note changes. If your game logic is simple and doesn't need a lot of CPU time, it might be a good option.

Re: In-game (during gameplay) music

cmgonzalez wrote:

Do you have any code example for that obscure solution that you mention?

No, if I had something, I sure would publish it already. At the moment, the only game I can think of that kind of used this approach was Sea Dragon port, where I did beeper effects. It used very crude sample-like effects, like every so often during rendering a frame the game just read a bit from an array and outputted it. As it was 50 hz targeted, rendering speed was relatively steady, thus 'sample rate' was steady enough too. I don't have this code around, and can't really recall how many times per frame the routine was called. I think it was the least possible number, to avoid introducing slowdowns. Maybe 8-16 updates per frame.

website - 1bit music - other music - youtube - bandcamp - patreon - twitter (latest news there)

12 (edited by cmgonzalez 2019-03-26 12:42:01)

Re: In-game (during gameplay) music

Thanks Utz n Shiru! But I also have desisted to do some fancy solutions, for the moment.

Beepfx have give me good results, I've to use it without disabling interrupts, by commenting di and ei instructions.

But the problem is the pitch, as the sound now have to deal with the interruptions and nirvana the playback is slower than is supposed to work on interrupt-less enviroment....

Do you know some way to fix it, besides changing the pitch of the samples?

13

Re: In-game (during gameplay) music

I don't think there is a way to fix it. Even changing the pitch will not make the samples play as intended.
Interrupts and beeper sound don't mix well. And iirc Nirvana's interrupts take up almost the entire frame.

Re: In-game (during gameplay) music

Thanks, I will add only some basic sounds...

15 (edited by introspec 2020-04-16 00:05:26)

Re: In-game (during gameplay) music

utz wrote:

I don't think there is any code for the method Shiru describes, because nobody has implemented it. The idea for continuous sound would be not to call the sound generator through an ISR (because 50 Hz is too slow to make anything but the lowest audible tones), but to interleave it with the multicolour code at regular intervals. The usual restrictions with contention apply there, so it would still be quite tricky.

Actually, I have implemented exactly this idea in my 1K intro "Splash" (2014). The beeper engine there is very simplistic, single voice, but this is purely due to the size limitations I was dealing with.

In fact, I think that it won't be a large overstatement to say that most of my beeper experiments were about trying to push the envelope of how beeper sound can be combined with other things.

Shiru's methods 1 and 2 can be found in quite a few old games with beeper music. I never liked the resulting sound, so I did not do much research on it. Hence, I am not sure if e.g. Manic Miner uses method 1 or method 2 (method 1 seems more likely, given the lack of vsync in the game). My beeper scroller in "Dat Fuzz" (2014) is using, I guess, a variation of the method 2, with one modification: the visuals occupy small fixed time remaining after playing something like 2.5 frames of sound. So, improving the quality of sound is easy - just ensure that the sound does not get interrupted much.

Like I said, method 3 in Shiru's list can be found in "Splash" (2014). At the same time, outside of beeper, I am pretty sure that at least some demos with digital sound use a similar approach (e.g. "Scroller" (2010) or "Atarin" (2018)). In fact, I'd say that the devil is in the implementation detail: visuals usually require v-sync, while beeper (or digi) need +- constant rate of updating, which does not fit well with updating visuals synchronized to vsync or, more generally, doing any non-trivial logic. Specific solutions of this clash of needs can be different and varied; what I did in Splash does not make me happy and I am currently experimenting with another approaches.

A good way to think about all these methods is to think about code responsible for visuals as something that "distorts" sound. Then your strategy to improve the sound quality would always be about something that reduces the distortion. The larger proportion of time you can spend on making the sound, the better sound quality you'd get. This applies to all three of Shiru's methods.

However, there are other methods for reducing the distortion. Instead of trying to reduce the "amplitude" of the distortion ("amplitude" in the PWM sense would be the duration of anything that is not generating sounds), you can change the dominant frequency with which distortion occurs. E.g. the scroller in "Dat Fuzz" operates on the basis of one scroll every 3 frames only partly because this was the only way to significantly change the proportion of time between the work of the beeper engine and the work of the visual code. It is also important that the base frequency of the resulting distortion becomes 18.5Hz (which cannot be heard) instead of the pretty obvious 50Hz produced by engines operating on the basis of single frames logic. If taken to the extreme, some games do all drawing between the rows in the beeper engine, which tends to mask quite well even pretty significant interruptions in sound. Of course, from the point of view of visuals extremely slow updates do not work very well.

My demo "Rain" (2016) was an attempt to see what happens at the other end of the spectrum. What would happen with the sound if the distortion was to operate at very high frequency? In "Rain", the visual code occupies every 4th iteration of the sound loop running at something like 12kHz. It does distort the sound, but the nature of the distortion is such as to create "spacey" sound, which seems a lot more acceptable that what methods 1 or 2 can provide you with. Of course, the type of code you need to write for this mindset is not very vsync-friendly and, in fact, is not very visual-effects friendly. However, you do get surprising amount of time for doing useful work beyond making the sound, so definitely consider this approach too.

16

Re: In-game (during gameplay) music

Good writeup, introspec. I'll add that in addition to the pin pulse method, the Squeeker method also allows for some deviations in timing, and both of these methods still work (albeit with noticable degration of sound quality and of course frequency range) at fairly low update rates, as demonstrated by nanobeep.

Re: In-game (during gameplay) music

Well, Octode XL used in Rain is not really a pin pulse method, not in the usual sense of the word, because pins are at least as wide as the period of discretization loop (and are often wider). Squeeker method is also a funny mixture of two other mixing approaches. I got behind your recent work on new engines (you make too many of them, mate!!! smile ), but I think that the overall question of which mixing method would be more suitable for in-game music is still very much open in my mind.

I.e., the principles I covered in my write-up have nothing to do with the method of mixing. OK, I based Rain on Octode XL because it has low discretization frequency, which made it easier to interleave audio and video codes at high frequency. You can probably make similar claim about the Squeeker engine. I do not think we have any other guidance in this case, do we?

18 (edited by FrankT 2020-12-11 00:56:30)

Re: In-game (during gameplay) music

I've just recently been looking for simple 2 channel engine with decay, for some in-game music, in a very simple game.

Been trying to cut QChan down to 2 channels, no drums, might be what I need. Small player and small music data.

Having been away from speccy and coding for some years, I'm a bit rusty at this. And I've never written sound routines before, just used what's available. Currently going through QChan with debugger to document the code so I know exactly what's going on.

Edit: Just seen utz tutorial on writing a routine, so I'm going to do that instead. Cheers utz, very helpful that...

19

Re: In-game (during gameplay) music

Welcome back, mate! Looking forward to what you will come up with. If you have any questions we're here to help, of course.
I think Hikaru's new AMP16k might be interesting for you to study as well. Generally, the "Squeaker" method used there is a good candidate for in-game music because its CPU requirements are not that high, and it tolerates some deviance in timing as well. Though pin-pulse like synthesis like in Qchan should work well, too.

Re: In-game (during gameplay) music

9 months later, and a little bit off-topic from the ZX:
I've done a lot of fiddling & tinkering on the VZ200 with trying an anything and everything approach with the various players that I've got working on the VZ. Randomly select a player and try to get something (ala animated graphics) working , just as a trial and error to see & take note of the differences between the engines to work out what is capable of driving a few moving graphics and what is not.
The VZ is somewhat nearly on par with the ZX specs. Many of the players that I've managed to port and get running on the VZ, of course, consume or seem to consume near 100% utilisation.
I've had success with quattropic with bouncing fake equaliser bars in both text mode as well as the VZ's "high" resolution which is 128x64. Of course this has slowed the music down a fraction - enough to notice, but still listenable.
Also I grabbed the ZX10 player and am using it in the background for an unfinished car moving game ; admittedly the sprite-type graphics are very few so far, but what I do have working reads the keyboard fine, moves the graphics incredibly fast and smooth with no noticeable loss in the quality or the speed of the music.
Both of these I have added code to where the players original check-for-keyboard-to-quit routine is. So far seems to work ok.

On the other hand, the likes of Phaser3 is so cycle intensive & specific (and sounding awesome!) that changing or adding anything at all causes sound issues straight away. Took me ages and many attempts to even get it working on the VZ - with my limited knowledge.

21

Re: In-game (during gameplay) music

The problem on Speccy isn't so much one of finding some free cycles to update graphics. In fact, in PFM and Phaser type engines, you can actually waste quite a few cycles with barely any drawbacks. However, on Speccy we have this thing called memory contention. Very simply said, if you write to or read from certain memory locations on the Speccy, additional wait states will be introduced. While it is possible to predict when those wait states occur, actually writing an engine that will work around these limitations is extremely challenging, so basically nobody bothers. AMP16 is currently about as good as it gets when it comes to running 1-bit code in contended memory.