cinera_handmade.network/cmuratori/hero/code/code165.hmml

[video output=day165 member=cmuratori stream_platform=twitch stream_username=handmade_hero project=code title="Fixing an Asset System Thread Bug" vod_platform=youtube id=9_jVu7RfP88 annotator=Miblo annotator=debiatan]
[0:50][Two possible plans for today:]
[1:01][a) Construction of sentences and paragraphs]
[2:09][b) Fixing a bug. Let's fix the bug]
[3:43][Explanation of the bug]
[5:58][How does the bug happen and how should we fix it?]
[7:53][One approach: Marking assets as used or not used]
[8:29][Cons of asset locking]
[9:34][Blackboard brainstorming]
[10:04][In-flight asset protection]
[11:13][Usage patterns of assets]
[13:21][Keeping an asset usage count and its associated problems]
[16:05][Introducing render_group generation IDs]
[20:02][Cons of this approach: Possible wrapping of the IDs]
[21:44][Maybe the wrapping is not a problem, after all]
[23:14][Some other complications of this approach]
[27:14][Feasibility of using generation IDs given the current state of our asset system code]
[30:45][Generation IDs make MoveHeaderToFront superfluous]
[32:00][Doubly-linked lists and threading don't play well]
[35:58][Blackboard explanation of some problems we will encounter]
[37:29][One possible solution: Queuing our MoveHeaderToFronts so that only one thread performs them]
[39:02][Another solution: Lock-free heap data structure]
[39:59][Yet another idea... that also poses some problems]
[42:48][We could also sort the generation ID-asset pairs separately]
[46:20][How about a multithreaded heap?]
[48:22][Coding the "generation IDs" approach]
[55:28][Updating the GenerationID]
[57:24][Compressing GetBitmap and GetSound together]
[1:01:50][Leaving the code in a working state (although we are still not finished)]
[1:03:57][Q&A][:speech]
[1:04:52][@Sharlock93][Hey Casey, can you do a 2-min recap of everything done today?]
[1:07:49][@andsz_][How often do you actually implement something from a paper?]
[1:08:45][@SvK151][What would be the worst case scenario if you had multiple threads trying to remove an asset? Couldn't you just wait till no thread was trying to remove the asset?]
[1:09:14][@Pseudonym73][One way to avoid the lock is to split the LRU list into two parts. Keep the most recent 25% of it in one partition and the least recent 75% in another. Only move an asset to the head of the list if it's in the least recent part]
[1:09:54][@JamesWidman][So IIUC, our problem is that "least recently used", in this context, means "least-recently-stopped-being-used-in-all-threads", and we can't really know that for certain without having some form of inter-thread communication.... Is that correct?]
[1:10:43][Meathead protein-powder action][quote 165]
[1:13:18][@Pseudonym73][You don't take the lock on hot assets most of the time]
[1:14:14][@Sharlock93][I haven't seen you drink any milk this stream, is that why you are tired?!]
[1:14:56][@EzyFool][You can avoid the genid gap checking list by keeping min/max/cnt rendergroup "done" ids, and freeing all assets <max when cnt = max - min, then reset min=max. Not the best if you want to free ASAP all the time. and the window gets large]
[1:17:45][@EzyFool][You know the min rendergroup that completed, but like you mentioned at the start, if they complete out of order, the min isn't...]
[1:18:31][@EzyFool][...correct when a gap occurs, so you need to track the interval and jump ahead when it fills. But maybe I'm just misunderstanding the situation]
[1:19:40][@Vargolindo][I'm at the Day 19 video (improving audio sync) and in order to get timeBeginPeriod to work I had to call it twice. Do you have any idea why?]
[1:21:07][@hitmanatee][Any update on what algorithm you are using to do laundry these days?]
[1:21:50][@EzyFool][Never mind, my keyboard & my internet seem to be dying, I'll wait 'til tomorrow]
[1:22:11][@Sharlock93][I think we should do a detour and make a simple ray tracer]
[1:22:43][Wind it down][:speech]
[/video]