1 2
5 6
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
Since probably all games desync with the best sound plugin (Eternal SPU), and all the rest that CAN sync sound glitchy, there is a dreadful workaround to get the best audio quality, sacrificing insane amounts of time. 1. Capture the video itself. 2. Capture audio with glitchy plugin. 3. Start capturing audio with Eternal SPU and figure out at what point the game desyncs. 4. Watch the movie again without capturing - with regular plugin, save a state just before the desync frame. 5. Start capturing with Eternal SPU - from the savestate made in the step above - till the next desync. 6. Get to the new desync place with normal plugin, make next savestate, and capture from it with Eternal SPU. 7. Repeat until you get all the movie captured, then stitch the segments and move their audio to the final encode. No one knows HOW OFTEN it would desync per TAS, depends on the game. For now, only Aktan uses the above madness and gets the best sound. We must work out what can be automated in this workflow, since it's TONS of routine no one is going to reward you for. - Fix the very plugin not to break everything. - Automate getting the desync frames. - Using 2 emulator instances - 1 with regular SPU for saving, another with Eternal SPU for loading and capturing. Can savestates be saved to files and then loaded on fly as PCSX gets to appropriate frames? For exampe, we make a LUA script that accepts frame numbers, saves states on them, and loads them when necessary during capturing? Would it lag if we load a state once in a while during kkapture? I mean, if we at first run the instance with Eternal SPU to get the desync points, and only then CAPTURE using all the savestates we made. Did anyone understand what I mean? :D EDIT: Dammit seems to have used some automation lua already: #2621: Dammit's PSX Bushido Blade 2 "Slash Mode" in 04:20.23 But the hosting site doesn't exist anymore.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Emulator Coder, Skilled player (1141)
Joined: 5/1/2010
Posts: 1217
When the BoF3 run was made, somebody wrote this lua script to make PCSX encoding easier: http://ompldr.org/vNzNtNw/encoding-bof3-lua.zip ... Dunno what it does exactly.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
Ilari wrote:
When the BoF3 run was made, somebody wrote this lua script to make PCSX encoding easier: http://ompldr.org/vNzNtNw/encoding-bof3-lua.zip ... Dunno what it does exactly.
This is marked as V1, here's V3:
BadPotato wrote:
I made some lua script over here, that might hopefully help you. Here the general idea about how it work out. First, there should be a bofCreateSaveState.lua, that pretty much produce a folder with lot of savestate, where there might be a desynch* . To create the savestate, just run the script with the TAS sound plugin. Once you got the savestate you can use the bofSynchHighQuality.lua script to playback the movie with Eternal sound plugin and kkapture it while running the script. Of course, these script aren't optimized and are quite messy, but feel free look around and edith them. I think that making the encode this way is more conveniant than manually playback thought the movie and kkapture the full movie in ~30 or ~50 parts for about each savestate used and then, merge them altogether. * I had to use 2 other script to produce an unoptimized "frame/checkpoint" list... I could post them if you're interested.
We need someone to modify it to make the tool universal and to write a good instruction.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Publisher
Joined: 4/23/2009
Posts: 1283
Dang, if I can automate parts of it, that be great, lol
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
Yep, I guess I'm the guys who tried to do this ^_^ The workflow described by feos is exactly what I tried to do, based on this theory(should be efficient 99% of the time): "When a movie desynch, the 'screen capture' should be different from the original movie". So first, I advice to use the "pcsxrr-13c" made by gocha, if you want to get the lua console windows. This build got removed for some reason when psxjin got released at the time... but I still kept it! (hehehe..^^) Here's just take it. Now, if I remember correctly(sorry for been so badly documented), ScreenshotTrack.lua will make a screenshot for every "n" frames(I setted n=10 to speed up the process). I got the screenshot with the gd library which actually use a String format to store the image(weird huh?), so since it's just a String I used the function "StringHash(text)" to convert the string as a smaller String. Then I store this info in hashFrame.txt. Later this script got merged with "CreateHQSaveState". So what's is bofCreateHQSaveStateV#.lua? I actually made a v4 and even an another version for suikoden2... but I resigned after a while, since this project grow bigger than I expected. My plan, with this script was to run the movie with the default pcsxrr TAS plugin, and create the savestate for every "checkpoint" frames. But how do you get the list of these checkpoints? So I created the "SynchHighQuality" script which should be used with the Eternal SPU plugin. When running this script, it should prompt "DESYNCH AT FRAME ... 'number' ", when the "StringHash" doesn't match. Now where it's beginning to get a bit annoying is that Eternal SPU might desynch according to a previous state of the emulator. So I added some other function to do some "savestate rollback", for debugging, etc. Overall, I couldn't figure out how to make this "workflow" fully automatic, so I still had to do multiple pass with these script and a lot of trial and error to get the "checkpoint" frame list. So yeah, if you understand my logic, once we get the "checkpoint list", its become very easy to use kkapture with the "SynchHighQuality" script and then do usual encoding stuff. Also, maybe we should contact Dammit, since his script isn't avalaible anymore and I actually didn't even know that I got thanked in his submission. I'll try to help, even if I got fewer time than I used to do. Someone is up to create a new "psx encoding workflow" project somewhere?
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
The new idea I got is to create REALLY MUCH savestates and screenshots from the original plugin, then actually run the screenshot comparison script with Eternal SPU and once it desyncs, load SOME state back. Then try running again and get the new desync frame. This may be optimized to actually DUMP only certain frames to load states on while kkapturing. So you get 3 passes only: 1. Dump savestates+screenshots (every 120 frames? dunno what would guarantee the proper check and not eat too much time. maybe the offsets for savestates and screenshots must be different). 2. Run the movie with eternal spu, get the desync frame, load the previous state, run again, check if it desyncs AT THE SAME FRAME, load the state farther back and verify again, etc. - till the end of movie. Dump FINAL frames when loading states fixed desync. 3. kkapture with loading the states at dumped frames. Each pass is automatic, it just needs to write another script :D
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
MESHUGGAH
Other
Skilled player (1888)
Joined: 11/14/2009
Posts: 1349
Location: 𝔐𝔞𝔤𝑦𝔞𝔯
I never used PCSX nor have any idea how tases desyncs with Eternal SPU recorded with other plugin, but shouldn't be possible to "detect" where's the desync point? I mean either by checkin the differences between the sound plugins and Eternal SPU or memory addresses for the game or for the PCSX? Also I don't know how big is the savestates for PCSX but you could literally dump all of them and then dump it with Eternal SPU and when a savestate is not the same with the other one then use the working "non-eternal spu" savestate and continue the job. I have a guess that this would take for many days for 1 TAS.
PhD in TASing 🎓 speedrun enthusiast ❤🚷🔥 white hat hacker ▓ black box tester ░ censorships and rules...
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
The problem is, you can't figure out ALL desync frames straight away, because you need to reload the state to return sync for the farther part. But this (resyncing and resuming) is already included in the script by badpotato. And about dumping ALL fames to savestates (aka Greenzone in TASEditor) is not actually needed. We only need to figure out the best frequency for that. Also, it's probably faster to compare screenshot hashes than memory states. So, basically, we just need some smart modification to the existing script to make it more readable and saving states at fixed freq to allow the easiest possible resync.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Publisher
Joined: 4/23/2009
Posts: 1283
Sounds like progress can be made in making it easier at least. I should mention that usually what I do is make a save state 15 frames before the known desync point. Comparing screenshots to see the exact desync point is a great idea, and if you can delete on fly after the compare, I think the space usage would be low.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
MESHUGGAH wrote:
Also I don't know how big is the savestates for PCSX but you could literally dump all of them and then dump it with Eternal SPU and when a savestate is not the same with the other one then use the working "non-eternal spu" savestate and continue the job. I have a guess that this would take for many days for 1 TAS.
The plugin does affect the memory at some random place in the background since any game might handle de sound in his own way, but that doesn't mean that an actual desynch happened. So we can't just look at the memory and try to estimate a reasonable comparaison from it.
feos wrote:
So, basically, we just need some smart modification to the existing script to make it more readable and saving states at fixed freq to allow the easiest possible resync.
I should point out that if we tried to use more checkpoint than necessary, since the game will be loading a state from the "normal TAS plugin", when replaying the final video you might get at some point some "weird sound", since the emulator have to deal from a previous memory state from an another plugin. As I said, sometime weird stuff might happen. If you're unlucky enough, I *think* the emulator could freeze or crash, but this kind of case should be kinda rare, but this is still a huge deal... and maybe we could even try to write some kind of code that detect this kind of case, as described here. Now about the right frequency for taking the screenshot, every frame is the safest, so I guess it's up to the encoder to decide this setting. Some game might have a huge deal of "desynch peek" where you might have to load several savestates in a very short amount of frame between them. And yes, the space usage for the screenshot is very low since we can produce some hashed data from it. Now about the frequency for the savestate, I'll say the bigger is the better for avoiding crash, maybe around ~5000frame. Also since the time for getting to the "Rollback state" isn't that big, the waiting time should be decent. But I guess we should be able to confirm these info with several experimentation.
feos wrote:
So you get 3 passes only: 1. Dump savestates+screenshots (every 120 frames? dunno what would guarantee the proper check and not eat too much time. maybe the offsets for savestates and screenshots must be different). 2. Run the movie with eternal spu, get the desync frame, load the previous state, run again, check if it desyncs AT THE SAME FRAME, load the state farther back and verify again, etc. - till the end of movie. Dump FINAL frames when loading states fixed desync. 3. kkapture with loading the states at dumped frames.
Right now, you might have to alternate between the passes 1 and the passes 2, if you want to optimize the checkpoint list and make sure everyting run fine. At some point, I even thought that it would be possible to run two instances of pcsxrr and make them communicate with data with text file and keep the workflow running. One instance would be running on the normal TAS plugin and the other run with the Eternal SPU. That might sound crazy, but in the end some processing time should saved on a multiple core CPU.(I think pcsxrr will alway use 100% of an single core). To manage this, either we use try to write kind of lua "communication manager" or we add an another kind of layer over that could be java or anything that handle the whole workflow. Anyway, I just wanted to give an idea of all the possibility. I think we should first try something that really work, before rushing on any other crazy/weird stuff. Since then, I'll try to review my lua scripting skill... Today I can't even read what I was thinking back then :)
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
You don't need to LOAD the state so much, it must only be tested how frequent saving it goes, and what we can afford without problems. BizHawk uses every frame savestate to support the Greenzone, that you can navigate back and forth throu - and thats even for SNES. It's unprobably that SAVING a state every second would crush anything. As for 2 instances - it also sounds quite awesome. You just need to time the actions between them so that while one is writing to the file, another one waits and then reads that file. Yesterday I tried to write a code for saving states with TAS plugin, but when it came to USING the saved data during the second pass I considered more experienced people to code it, not because I can't do it (I can), but because it's better if the author modifies his own script sometimes. I will try running the savestate code to see how it works.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
One more thing I forgot to mention, kkapture crash after it capture around 2gb. So for very long movie, the encoder still have to do some manual stuff between each part. So it's kind of annoying, also if I remember well there is no command line feature for kkapture or maybe didn't look on this correctly? The size of the encoding file could probably be monitored by script or we might setup a maximum frame for each part, so at the worst case if there NO way to get around of the 2gb limitation, I could prompt a console message from windows. This means, that kkapture will be suspended as long as the prompt message isn't closed. I already did that in the past and it work, but I'm sure there a better solution for this particuliar problem.
creaothceann
He/Him
Editor
Joined: 4/7/2005
Posts: 1874
Location: Germany
The source is available, so someone could add splitting...
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
All kkapture I used splitted the files at around 1.8 GB.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Publisher
Joined: 4/23/2009
Posts: 1283
Yea, that is weird, I do 120 GB captures with .kkapture with it auto splitting... Also, command line to the forked application is there, but command line to .kkapture itself isn't there.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
Ok, actually for the split issue I believe this was the sound or the video that ended up been corrupted. But, I'll try to do more test about it later to confirm this... As for the command line for kkapture, I made a script with autohotkey so I can get some easy macro to work on. It think it work a bit like the windows wscript, but anyway. As promised here a new version of my old workflow with some refactoring and code clean up! Get it here : PcsxrrEncodeWofkflowV4 ! (the autohotkey script is there as well) The "detectCheckpoint.lua" and "synchCheckpoint.lua" have some default data from my old suikoden II speedrun, but feel free to experiment with your own movie and give me some feedback. Right now, this isn't really more automated than my previous script. I think there still a bug when loading a savestate and comparing screenshot. So I'll make sure to look at this sometime later. Also, more I'm thinking about it, more I think that doing the whole process with 2 pcsxrr instance is the best way to go. But I'm still unsure how I'm going to figure how I should manage communication, update, etc... between these two script.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
I'm somewhat stuck, when loading a savestate the screen isn't updated right away. I would badly need some redraw function :( If we ignore this problem, sometime the previous frame when loading the savestate might be duplicated... kinda weird. The only way to "update perfectly" without skip from any "state" of the emulator(example you want to loadstate frame 1000 from frame 1050), is to pause the emulator, load the savestate and give the control to the user. So I've also tried to find if there any other "hackish" way to update the screen without giving the control to the user, but here what I found while messing around: There's a bug with pcsx.unpause(), that make it exiting the script. But I found out that if you do a pcsx.pause() or a pcsx.unpause() inside a "lua corrutine", the main script will continue to run. Sadly, both function still work the sameway has pcsx.frameadvance(), so the screen isn't updated. Anyway, messing with the lua corrutine made me realize that they work in a very different way as the normal thread in other programming language... they almost work as normal function. A possible plan to get rid of this problem would be to try an another multitasking lib to create a separate thread that will give the control to the user in order to update the screen and then go back to the main lua script. But I think that might become a bit complicated. I guess what I said might sound like gibberish, but this isn't really easy to describe either. On another hand if we get a duplicated frame on every "savestate checkpoint", that could be an actual fair trade for getting a fully automated process. So I might as well ignore this problem since I believe it could be fairly easy to just replace the correct frame by the right screenshot in the final encoding process. Meanwhile, here's an update : PcsxrrEncodeWofkflowV4.01 I added a configuration/propreties files in lua, by using the lua module feature. I really don't know if this is the right way to it, but at least I don't have to repeat the same info at multiple place.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
We DO NOT need to capture VIDEO when capturing the final AUDIO stream. Video is supposed to be captured before, with whichever audio plugin that syncs.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Publisher
Joined: 4/23/2009
Posts: 1283
feos wrote:
We DO NOT need to capture VIDEO when capturing the final AUDIO stream. Video is supposed to be captured before, with whichever audio plugin that syncs.
This is incorrect for certain graphics plugins, like TAS Soft Graphic Plugin. There are missing load frames from the video if you don't use Eternal SPU and used a sync audio plugin.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
Alright, I figured that I should probably post an update, since there probably some people waiting for the script to be finished and the new PSX Mega Man X6(or Aktan already made it?) run seem to be waiting for his encode and it wouldn't be acceptable to keep relaying on time-consumming way to produce encode again. I really don't know how I'll manage my time, so I can't garanteed how long this gonna take. I'm just a dude that got way to many stuff to do... but I'll still try my best to help out. So here's an update: After serials severals experiments with the Lanes library, I managed to make a function that redraw the screen. Sadly it might not been used in the syncCheckpoints script since I found an another and easier way get rid of the frame duplication problem without having to relay on lua multi-threading. But since, I already made the code for that redraw function, I'll still leave it there in the project and it might still been used for the double instance setup. As I said, I think the double setup is the only way to get an "automatic workflow". Thankfully, the script doesn't have that much work left to do. I basically have all the piece of the puzzle... I just need to be put them together and test the whole thing. For the communication between the pcsxrr instance, I ended up going for the LuaXML API, that give some basic tool for both writing/parsing a status file. One status file should be used for each pcsxrr instance and at some point one instance would send a request for the other instance to produce a specific checkpoint, etc.. Using XML should work, but I've been interested by the lua ØMQ API, as well. It offer to send data by using socket instead of using file for I/O. I really don't know if that could have been working correctly, thought. Also, I just remembered something that could be terribly annoying. If my memory's right, sometime the Eternal SPU plugin will desync more or less often while using kkapture in same time. I'll need to investigate more about this, since I'm still unsure how I used to cause such issue the happen and if this is avoidable(maybe forcing the pcsxrr speed to normal?). As for the video plugin issue pointed by Aktan for the loading screen(I guess this is what he mean), I gonna admit that this confuse me quite a bit, since so far I relay a lot on the "TAS Soft Graphics plugin 0.2" to produce accurate screenshot with the "TAS Sound Plugin 0.2". The final "checkpoint list" of savestate might then been innacurate if the screenshot are somewhat corrupted, but think this problem is more relevant as an emulation and there's a limit on what we can fix in the encoder corner. That's said, that made me thinking that there also the "TAS OpenGL Graphics Plugin 0.1" that is supposed work better for few game. I guess it could be interesting to see if the plugin could be used partly or completely for some encode... Update: PcsxrrEncodeWofkflowV4.07 If anyone is interested, I guess I could make some Google project or GitHub repo for this.
Publisher
Joined: 4/23/2009
Posts: 1283
Thanks a lot for your work BadPotato. As for PSX MegaMan X6, I've already found all the desyncs and encoded the video. It will be up soon (I hope). When I get some time, I will try out what you made, BadPotato.
Spikestuff
They/Them
Editor, Publisher, Expert player (2283)
Joined: 10/12/2011
Posts: 6335
Location: The land down under.
I really like this. I was thinking of encoding some older tas runs (Webnation uploads) but some actually gave me deysnc (eg. Umihara Kawase Shun Desyncs at the start). So I'm very pleased about seeing a project that will eliminate any desync.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11264
Location: RU
Aktan wrote:
feos wrote:
We DO NOT need to capture VIDEO when capturing the final AUDIO stream. Video is supposed to be captured before, with whichever audio plugin that syncs.
This is incorrect for certain graphics plugins, like TAS Soft Graphic Plugin. There are missing load frames from the video if you don't use Eternal SPU and used a sync audio plugin.
Then we are not able to compare screenshots because the frames don't match. Are we?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Publisher
Joined: 4/23/2009
Posts: 1283
feos wrote:
Aktan wrote:
feos wrote:
We DO NOT need to capture VIDEO when capturing the final AUDIO stream. Video is supposed to be captured before, with whichever audio plugin that syncs.
This is incorrect for certain graphics plugins, like TAS Soft Graphic Plugin. There are missing load frames from the video if you don't use Eternal SPU and used a sync audio plugin.
Then we are not able to compare screenshots because the frames don't match. Are we?
I think it will still match. It isn't like the capture with the missing frames have new frames. They just have duplicate frames of the last frame before the missing frames start as spacers. For example, a capture with missing frames would have the frame counter go like so: 0, 1, 2, 3, 10, 11, ... while a capture without missing frames would have the frame counter go like so: 0, 1, 2, 3, 3, 3, 3, 3, 3, 3, 10, 11, .... I hope I explained this clearly.
Joined: 1/26/2009
Posts: 558
Location: Canada - Québec
I think the run with the must "loading fest" happening is probably Azure Dreams. I guess we could compare the current encode and the "luascript" encode later, so in the end we should be able to know if there anything going different. In theory this shouldn't be the case. Allow me to recap with my low redaction skill, I'm sure this isn't necessary, but at the worse case, this is just gonna to boost my mind focus... hm yeah, that's right :) The "detection script"(TAS sound plugin) automate the process for getting specific frame know as checkpoint in the run, that might desync with Eternal SPU sound plugin(best sound quality). These checkpoint are taken as savestate with TAS sound plugin(best sync, but terrible sound quality). Also that very same script capture screenshot and convert it as a number, just like a md5sum program Then the "sync script"(Eternal SPU plugin) use the"savestate checkpoints"(from the TAS sound plugin) as a way to play thought the movie without getting any desync, while checking if everyframe is fine with the screenshot. As said previously, there shouldn't be any missing frame from the duplicated frame that might happen on loading screen. Now some of you might wonder about the process and say: Hey potato! Why would you not use a savestate from the TAS plugin for every frame and just play the whole thing? Well, the reason isn't because it might use of GB of storage, but is the following. Everytime we load a savestate from the TAS sound plugin, we still heard a quick sound output of the original savestate(with the TAS sound plugin) for that 1 frame. This means, that if there to many savestate... that sound will be audible, so we might want to optimize the checkpoint list, depending on how often a game might desync. To optimize this list, there are various trick, that I might not implement. But here's one: Sometime the sound isn't exactly the cause of the desync. I'm not sure what this is about, but I found out that if you simply reload a savestate from a couple of frame earlier before the desync frame with the Eternal SPU plugin, you can actualy avoid a desync! Since reloading a savestate from the Eternal plugin doesn't produce any "sound glitch", I might consider storing Eternal sound plugin savestate as well. Hehe, did anyone even got what I mean? So yeah, I guess it's possible to go crazy for trying to optimize the checkpoint workflow, but as I said I'll just keep this script simple stupid and just try to get this working. Can't garanty how ugly the script will be, thought. At some point, I could try some "simple" OO approach, but that's kinda a tricky matter, since there several way to create "object" in lua. Sadly I didn't saw anyone on this website, trying to put some effort on making any sort of common lib or framework project for Lua TASing, so I really wonder what's up with that and if should try to make some base as an alternative project to make some class here and there, instead of using module. But, I guess this isn't a big deal either. I'll post an another update soon. Also my last script had several error :/
1 2
5 6