Annotation-Pushers
/
Annotation-System
1
Fork 0
Code Issues 1 Pull Requests Projects Releases Wiki Activity

cinera.c: Remove spurious file I/O 

Don't \-escape anything in the index

Document server's security header (recommended) requirements
This commit is contained in:
Matt Mascarenhas 2018-05-01 22:05:44 +01:00
parent be37ea234a
commit e77d208645
3 changed files with 234 additions and 313 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
README.md
View File

@ -11,9 +11,9 @@ deployment
4. `cp hmml.a hmmlib.h ../cinera/` 4. `cp hmml.a hmmlib.h ../cinera/`
5. `cd ../cinera/` 5. `cd ../cinera/`
Note: For each parser update, remember to make and copy it into place Note: For each parser update, remember to make and copy it into place.
### Install Dependencies ### Install the dependency
1. curl 1. curl
@ -21,6 +21,20 @@ Note: For each parser update, remember to make and copy it into place
1. `$SHELL cinera.c` 1. `$SHELL cinera.c`
### Configure the server
If you enforce a strict Content Security Policy and X-Frame-Options in your
server configuration as recommended by [Security
Headers](https://securityheaders.com/), you may enable _Cinera_ to function by
making two small tweaks:
add_header Content-Security-Policy "default-src … https://www.youtube.com https://s.ytimg.com";
add_header X-Frame-Options "ALLOW-FROM https://www.youtube.com";
Note: For more information about these and other security headers, see Scott
Helme's articles [Content Security Policy - An Introduction](https://scotthelme.co.uk/content-security-policy-an-introduction/)
and [Hardening your HTTP response headers](https://scotthelme.co.uk/hardening-your-http-response-headers/#x-frame-options).
### Run ### Run
#### Single Edition operation #### Single Edition operation
@ -28,14 +42,14 @@ Note: For each parser update, remember to make and copy it into place
cinera test.hmml cinera test.hmml
This simply generates an HTML file (and updates `cinera_topics.css` if needed) This simply generates an HTML file (and updates `cinera_topics.css` if needed)
from `test.hmml` and outputs to `out.html` from `test.hmml` and outputs to `out.html` (configurable with -o).
#### Project Edition operation #### Project Edition operation
cinera -p ProjectID cinera -p ProjectID
Setting the ProjectID with the `-p` flag triggers Project Edition. In this Setting the ProjectID with the `-p` flag triggers Project Edition. In this
edition `cinera` monitors the Project Input Directory for new, edited and edition _Cinera_ monitors the Project Input Directory for new, edited and
deleted .hmml files, and generates one table of contents / search page and a deleted .hmml files, and generates one table of contents / search page and a
player page each for valid sets of annotations (or removes them, if needed). player page each for valid sets of annotations (or removes them, if needed).
@ -93,7 +107,12 @@ directory. Typical operation will involve setting these flags:
hold up to 255 characters 12 to 15 may hold up to 1023 characters_ hold up to 255 characters 12 to 15 may hold up to 1023 characters_
Feel free to play with templates to your heart's content. If you do anything Feel free to play with templates to your heart's content. If you do anything
invalid, `cinera` will tell you what's wrong. invalid, _Cinera_ will tell you what's wrong.
Note: There is currently an issue sizing the annotation marker container if your
HTML document contains `<!DOCTYPE html>`. To work around this for now, please do
not declare a DOCTYPE and instead let the page be displayed in [quirks
mode](https://www.w3.org/International/articles/serving-xhtml/#quirks).
#### Arguments #### Arguments
@ -123,6 +142,7 @@ invalid, `cinera` will tell you what's wrong.
-m <default medium> -m <default medium>
Override default default medium ("programming") Override default default medium ("programming")
Known project defaults: Known project defaults:
bitwise: programming
book: research book: research
pcalc: programming pcalc: programming
riscy: programming riscy: programming
@ -179,6 +199,10 @@ invalid, `cinera` will tell you what's wrong.
Display (examine) index file and exit Display (examine) index file and exit
-f -f
Force integration with an incomplete template Force integration with an incomplete template
-g
Ignore video privacy status
NOTE: For use with projects whose videos are known to all be public,
to save us having to check their privacy status
-w -w
Force quote cache rebuild (memory aid: "wget") Force quote cache rebuild (memory aid: "wget")

415
cinera/cinera.c
View File

@ -16,7 +16,7 @@ typedef struct
version CINERA_APP_VERSION = { version CINERA_APP_VERSION = {
.Major = 0, .Major = 0,
.Minor = 5, .Minor = 5,
.Patch = 46 .Patch = 47
}; };
// TODO(matt): Copy in the DB 3 stuff from cinera_working.c // TODO(matt): Copy in the DB 3 stuff from cinera_working.c
@ -207,7 +207,7 @@ typedef struct
char BaseURL[MAX_BASE_URL_LENGTH + 1]; char BaseURL[MAX_BASE_URL_LENGTH + 1];
char IndexLocation[MAX_RELATIVE_PAGE_LOCATION_LENGTH + 1]; char IndexLocation[MAX_RELATIVE_PAGE_LOCATION_LENGTH + 1];
char PlayerLocation[MAX_RELATIVE_PAGE_LOCATION_LENGTH + 1]; char PlayerLocation[MAX_RELATIVE_PAGE_LOCATION_LENGTH + 1];
char PlayerURLPrefix[MAX_PLAYER_URL_PREFIX_LENGTH + 1]; char PlayerURLPrefix[MAX_PLAYER_URL_PREFIX_LENGTH + 1]; // TODO(matt): Replace this with the OutputPath, when we add that
} index_header; } index_header;
typedef struct typedef struct
@ -417,8 +417,8 @@ typedef struct
credential_info Credentials[] = credential_info Credentials[] =
{ {
{ "/a_waterman", "Andrew Waterman", "https://www.linkedin.com/in/andrew-waterman-76805788", "", ""}, { "a_waterman", "Andrew Waterman", "https://www.linkedin.com/in/andrew-waterman-76805788", "", ""},
{ "/y_lee", "Yunsup Lee", "https://www.linkedin.com/in/yunsup-lee-385b692b/", "", ""}, { "y_lee", "Yunsup Lee", "https://www.linkedin.com/in/yunsup-lee-385b692b/", "", ""},
{ "AndrewJDR", "Andrew Johnson", "", "", ""}, { "AndrewJDR", "Andrew Johnson", "", "", ""},
{ "AsafGartner", "Asaf Gartner", "", "", ""}, { "AsafGartner", "Asaf Gartner", "", "", ""},
{ "BretHudson", "Bret Hudson", "http://www.brethudson.com/", "cinera_sprite_patreon.png", "https://www.patreon.com/indieFunction"}, { "BretHudson", "Bret Hudson", "http://www.brethudson.com/", "cinera_sprite_patreon.png", "https://www.patreon.com/indieFunction"},
@ -726,33 +726,6 @@ CopyStringToBufferHTMLSafeBreakingOnSlash_(int LineNumber, buffer *Dest, char *S
} }
} }
#define CopyStringToBufferJSONSafe(Dest, String) CopyStringToBufferJSONSafe_(__LINE__, Dest, String)
void
CopyStringToBufferJSONSafe_(int LineNumber, buffer *Dest, char *String)
{
char *Start = String;
int Length = StringLength(String);
while(*String)
{
switch(*String)
{
case '\\':
case '\"':
*Dest->Ptr++ = '\\';
++Length;
default:
*Dest->Ptr++ = *String++;
break;
}
}
if(Dest->Ptr - Dest->Location >= Dest->Size)
{
fprintf(stderr, "CopyStringToBufferJSONSafe(%s) call on line %d cannot accommodate %d(+1)-character JSON-sanitised string:\n"
"%s\n", Dest->ID, LineNumber, Length, Start);
__asm__("int3");
}
}
#define CopyBuffer(Dest, Src) CopyBuffer_(__LINE__, Dest, Src) #define CopyBuffer(Dest, Src) CopyBuffer_(__LINE__, Dest, Src)
void void
CopyBuffer_(int LineNumber, buffer *Dest, buffer *Src) CopyBuffer_(int LineNumber, buffer *Dest, buffer *Src)
@ -770,6 +743,22 @@ CopyBuffer_(int LineNumber, buffer *Dest, buffer *Src)
*Dest->Ptr = '\0'; *Dest->Ptr = '\0';
} }
#define CopyBufferSized(Dest, Src, Size) CopyBufferSized_(__LINE__, Dest, Src, Size)
void
CopyBufferSized_(int LineNumber, buffer *Dest, buffer *Src, int Size)
{
Src->Ptr = Src->Location;
while(Src->Ptr - Src->Location < Size)
{
*Dest->Ptr++ = *Src->Ptr++;
}
if(Dest->Ptr - Dest->Location >= Dest->Size)
{
fprintf(stderr, "CopyBufferNoNull(%s) call on line %d cannot accommodate %d(+1)-character %s\n", Dest->ID, LineNumber, StringLength(Src->Location), Src->ID);
__asm__("int3");
}
}
int int
StringsDiffer(char *A, char *B) // NOTE(matt): Two null-terminated strings StringsDiffer(char *A, char *B) // NOTE(matt): Two null-terminated strings
{ {
@ -2628,7 +2617,7 @@ MediumExists(char *Medium)
int int
ReadFileIntoBuffer(file_buffer *File, int BufferPadding) ReadFileIntoBuffer(file_buffer *File, int BufferPadding)
{ {
if(!(File->Handle = fopen(File->Path, "r"))) if(!(File->Handle = fopen(File->Path, "r"))) // TODO(matt): Fuller error handling
{ {
return RC_ERROR_FILE; return RC_ERROR_FILE;
} }
@ -2954,6 +2943,7 @@ HMMLToBuffers(buffers *CollationBuffers, template **BespokeTemplate, char *Filen
RewindBuffer(&CollationBuffers->Player); RewindBuffer(&CollationBuffers->Player);
RewindBuffer(&CollationBuffers->ScriptPlayer); RewindBuffer(&CollationBuffers->ScriptPlayer);
RewindBuffer(&CollationBuffers->IncludesIndex); RewindBuffer(&CollationBuffers->IncludesIndex);
RewindBuffer(&CollationBuffers->Search);
*CollationBuffers->Custom0 = '\0'; *CollationBuffers->Custom0 = '\0';
*CollationBuffers->Custom1 = '\0'; *CollationBuffers->Custom1 = '\0';
*CollationBuffers->Custom2 = '\0'; *CollationBuffers->Custom2 = '\0';
@ -3274,7 +3264,6 @@ HMMLToBuffers(buffers *CollationBuffers, template **BespokeTemplate, char *Filen
if(Config.Edition != EDITION_SINGLE && !PrivateVideo) if(Config.Edition != EDITION_SINGLE && !PrivateVideo)
{ {
RewindBuffer(&CollationBuffers->Search);
CopyStringToBuffer(&CollationBuffers->Search, "name: \""); CopyStringToBuffer(&CollationBuffers->Search, "name: \"");
if(StringsDiffer(Config.PlayerURLPrefix, "")) if(StringsDiffer(Config.PlayerURLPrefix, ""))
{ {
@ -3288,7 +3277,7 @@ HMMLToBuffers(buffers *CollationBuffers, template **BespokeTemplate, char *Filen
CopyStringToBuffer(&CollationBuffers->Search, "\"\n" CopyStringToBuffer(&CollationBuffers->Search, "\"\n"
"title: \""); "title: \"");
CopyStringToBufferJSONSafe(&CollationBuffers->Search, HMML.metadata.title); CopyStringToBufferNoFormat(&CollationBuffers->Search, HMML.metadata.title);
CopyStringToBuffer(&CollationBuffers->Search, "\"\n" CopyStringToBuffer(&CollationBuffers->Search, "\"\n"
"markers:\n"); "markers:\n");
} }
@ -3681,12 +3670,12 @@ AppendedIdentifier:
if(Anno->is_quote && !Anno->text[0]) if(Anno->is_quote && !Anno->text[0])
{ {
CopyStringToBuffer(&CollationBuffers->Search, "\u201C"); CopyStringToBuffer(&CollationBuffers->Search, "\u201C");
CopyStringToBufferJSONSafe(&CollationBuffers->Search, QuoteInfo.Text); CopyStringToBufferNoFormat(&CollationBuffers->Search, QuoteInfo.Text);
CopyStringToBuffer(&CollationBuffers->Search, "\u201D"); CopyStringToBuffer(&CollationBuffers->Search, "\u201D");
} }
else else
{ {
CopyStringToBufferJSONSafe(&CollationBuffers->Search, Anno->text); CopyStringToBufferNoFormat(&CollationBuffers->Search, Anno->text);
} }
CopyStringToBuffer(&CollationBuffers->Search, "\"\n"); CopyStringToBuffer(&CollationBuffers->Search, "\"\n");
} }
@ -4833,55 +4822,36 @@ GetNeighbourhood(index *Index, neighbourhood *N, int IndexEditType, bool *ThisIs
int int
InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, template **BespokeTemplate, char *BaseFilename, bool RecheckingPrivacy) InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, template **BespokeTemplate, char *BaseFilename, bool RecheckingPrivacy)
{ {
int IndexMetadataFileReadCode = ReadFileIntoBuffer(&Index->Metadata, 0);
switch(IndexMetadataFileReadCode)
{
case RC_ERROR_MEMORY:
return RC_ERROR_MEMORY;
case RC_ERROR_FILE:
case RC_SUCCESS:
break;
}
int IndexFileReadCode = ReadFileIntoBuffer(&Index->File, 0);
switch(IndexFileReadCode)
{
case RC_ERROR_MEMORY:
return RC_ERROR_MEMORY;
case RC_ERROR_FILE:
case RC_SUCCESS:
break;
}
int IndexEntryInsertionStart = -1; int IndexEntryInsertionStart = -1;
int IndexEntryInsertionEnd = -1; int IndexEntryInsertionEnd = -1;
Index->Header.EntryCount = 0; Index->Header.EntryCount = 0;
ClearEntry(&Index->Entry); ClearEntry(&Index->Entry);
bool Reinserting = FALSE; bool Reinserting = FALSE;
if(IndexMetadataFileReadCode == RC_SUCCESS && IndexFileReadCode == RC_SUCCESS) index_metadata *Entry = { 0 };
if(Index->Metadata.FileSize > 0 && Index->File.FileSize > 0)
{ {
// TODO(matt): Index validation? // TODO(matt): Index validation?
// Maybe at least if(!StringsDiffer(..., "name: \""); // Maybe at least if(!StringsDiffer(..., "name: \"");
// and check that we won't crash through the end of the file when skipping to the next entry // and check that we won't crash through the end of the file when skipping to the next entry
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location; Index->Header = *(index_header *)Index->Metadata.Buffer.Location;
Index->Header = *(index_header *)Index->Metadata.Buffer.Ptr;
Index->Header.CurrentDBVersion = CINERA_DB_VERSION; Index->Header.CurrentDBVersion = CINERA_DB_VERSION;
Index->Header.CurrentAppVersion = CINERA_APP_VERSION; Index->Header.CurrentAppVersion = CINERA_APP_VERSION;
Index->Header.CurrentHMMLVersion.Major = hmml_version.Major; Index->Header.CurrentHMMLVersion.Major = hmml_version.Major;
Index->Header.CurrentHMMLVersion.Minor = hmml_version.Minor; Index->Header.CurrentHMMLVersion.Minor = hmml_version.Minor;
Index->Header.CurrentHMMLVersion.Patch = hmml_version.Patch; Index->Header.CurrentHMMLVersion.Patch = hmml_version.Patch;
Index->Metadata.Buffer.Ptr += sizeof(index_header); *(index_header *)Index->Metadata.Buffer.Location = Index->Header;
Index->File.Buffer.Ptr += StringLength("---\n"); Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(Index->Header);
Index->File.Buffer.Ptr = Index->File.Buffer.Location + StringLength("---\n");
index_metadata *Entry = { 0 };
N->ThisIndex = BinarySearchForMetadataEntry(Index, &Entry, BaseFilename); N->ThisIndex = BinarySearchForMetadataEntry(Index, &Entry, BaseFilename);
if(Entry) if(Entry)
{ {
// Reinsert // Reinsert
Reinserting = TRUE; Reinserting = TRUE;
N->This.Size = Entry->Size;
N->This.LinkOffsets.PrevStart = Entry->LinkOffsets.PrevStart; N->This.LinkOffsets.PrevStart = Entry->LinkOffsets.PrevStart;
N->This.LinkOffsets.PrevEnd = Entry->LinkOffsets.PrevEnd; N->This.LinkOffsets.PrevEnd = Entry->LinkOffsets.PrevEnd;
N->This.LinkOffsets.NextStart = Entry->LinkOffsets.NextStart; N->This.LinkOffsets.NextStart = Entry->LinkOffsets.NextStart;
@ -4911,17 +4881,11 @@ InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, templ
else else
{ {
// NOTE(matt): Initialising new index_header // NOTE(matt): Initialising new index_header
Index->Header.CurrentDBVersion = CINERA_DB_VERSION; Index->Header.InitialDBVersion = Index->Header.CurrentDBVersion = CINERA_DB_VERSION;
Index->Header.CurrentAppVersion = CINERA_APP_VERSION; Index->Header.InitialAppVersion = Index->Header.CurrentAppVersion = CINERA_APP_VERSION;
Index->Header.CurrentHMMLVersion.Major = hmml_version.Major; Index->Header.InitialHMMLVersion.Major = Index->Header.CurrentHMMLVersion.Major = hmml_version.Major;
Index->Header.CurrentHMMLVersion.Minor = hmml_version.Minor; Index->Header.InitialHMMLVersion.Minor = Index->Header.CurrentHMMLVersion.Minor = hmml_version.Minor;
Index->Header.CurrentHMMLVersion.Patch = hmml_version.Patch; Index->Header.InitialHMMLVersion.Patch = Index->Header.CurrentHMMLVersion.Patch = hmml_version.Patch;
Index->Header.InitialDBVersion = CINERA_DB_VERSION;
Index->Header.InitialAppVersion = CINERA_APP_VERSION;
Index->Header.InitialHMMLVersion.Major = hmml_version.Major;
Index->Header.InitialHMMLVersion.Minor = hmml_version.Minor;
Index->Header.InitialHMMLVersion.Patch = hmml_version.Patch;
CopyStringNoFormat(Index->Header.ProjectID, sizeof(Index->Header.ProjectID), Config.ProjectID); CopyStringNoFormat(Index->Header.ProjectID, sizeof(Index->Header.ProjectID), Config.ProjectID);
@ -4958,7 +4922,7 @@ InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, templ
switch(HMMLToBuffers(CollationBuffers, BespokeTemplate, InputFile, N)) switch(HMMLToBuffers(CollationBuffers, BespokeTemplate, InputFile, N))
{ {
// TODO(matt): Actually sort out the fatality of these cases, once we are always-on // TODO(matt): Actually sort out the fatality of these cases
case RC_ERROR_FILE: case RC_ERROR_FILE:
case RC_ERROR_FATAL: case RC_ERROR_FATAL:
return RC_ERROR_FATAL; return RC_ERROR_FATAL;
@ -4971,48 +4935,36 @@ InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, templ
VideoIsPrivate = TRUE; VideoIsPrivate = TRUE;
case RC_SUCCESS: case RC_SUCCESS:
break; break;
}; }
ClearCopyStringNoFormat(N->This.BaseFilename, sizeof(N->This.BaseFilename), BaseFilename); ClearCopyStringNoFormat(N->This.BaseFilename, sizeof(N->This.BaseFilename), BaseFilename);
if(N->This.Size > 0) if(N->This.Size > 0) { ClearCopyStringNoFormat(N->This.Title, sizeof(N->This.Title), CollationBuffers->Title); }
{
ClearCopyStringNoFormat(N->This.Title, sizeof(N->This.Title), CollationBuffers->Title);
}
if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { FreeBuffer(&Index->Metadata.Buffer); return RC_ERROR_FILE; }
if(!(Index->File.Handle = fopen(Index->File.Path, "w"))) { FreeBuffer(&Index->File.Buffer); return RC_ERROR_FILE; }
if(!Reinserting) { ++Index->Header.EntryCount; }
fwrite(&Index->Header, sizeof(index_header), 1, Index->Metadata.Handle);
if(IndexMetadataFileReadCode == RC_SUCCESS)
{
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header);
}
if(Reinserting) if(Reinserting)
{ {
// NOTE(matt): We hit this during the start-up sync and when copying in a .hmml file over an already existing one, but
// would need to fool about with the inotify event processing to get to this branch in the case that saving
// a file triggers an IN_DELETE followed by an IN_CLOSE_WRITE event
//
// We will almost definitely need to handle our inotify events differently once we are outputting the player
// pages of private videos to "secret locations", because we will need to preserve the randomly generated
// location stored in the .metadata
// Reinsert // Reinsert
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->ThisIndex; if(!VideoIsPrivate)
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->This; {
fwrite(Index->Metadata.Buffer.Location + sizeof(index_header), Index->Metadata.FileSize - sizeof(index_header), 1, Index->Metadata.Handle); if(!(Index->File.Handle = fopen(Index->File.Path, "w"))) { return RC_ERROR_FILE; }
fwrite(Index->File.Buffer.Location, IndexEntryInsertionStart, 1, Index->File.Handle); fwrite(Index->File.Buffer.Location, IndexEntryInsertionStart, 1, Index->File.Handle);
fwrite(CollationBuffers->Search.Location, N->This.Size, 1, Index->File.Handle); fwrite(CollationBuffers->Search.Location, N->This.Size, 1, Index->File.Handle);
fwrite(Index->File.Buffer.Location + IndexEntryInsertionEnd, Index->File.FileSize - IndexEntryInsertionEnd, 1, Index->File.Handle); fwrite(Index->File.Buffer.Location + IndexEntryInsertionEnd, Index->File.FileSize - IndexEntryInsertionEnd, 1, Index->File.Handle);
fclose(Index->File.Handle);
if(VideoIsPrivate) if(N->This.Size == IndexEntryInsertionEnd - IndexEntryInsertionStart)
{ {
if(!RecheckingPrivacy) Index->File.Buffer.Ptr = Index->File.Buffer.Location + IndexEntryInsertionStart;
CopyBufferSized(&Index->File.Buffer, &CollationBuffers->Search, N->This.Size);
}
else
{
FreeBuffer(&Index->File.Buffer);
ReadFileIntoBuffer(&Index->File, 0);
}
LogError(LOG_NOTICE, "Reinserted %s - %s", BaseFilename, CollationBuffers->Title);
fprintf(stderr, "\e[1;33mReinserted\e[0m %s - %s\n", BaseFilename, CollationBuffers->Title);
}
else if(!RecheckingPrivacy)
{ {
LogError(LOG_NOTICE, "Privately Reinserted %s", BaseFilename); LogError(LOG_NOTICE, "Privately Reinserted %s", BaseFilename);
fprintf(stderr, "\e[0;34mPrivately Reinserted\e[0m %s\n", BaseFilename); fprintf(stderr, "\e[0;34mPrivately Reinserted\e[0m %s\n", BaseFilename);
@ -5020,11 +4972,14 @@ InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, templ
} }
else else
{ {
LogError(LOG_NOTICE, "Reinserted %s - %s", BaseFilename, CollationBuffers->Title); ++Index->Header.EntryCount;
fprintf(stderr, "\e[1;33mReinserted\e[0m %s - %s\n", BaseFilename, CollationBuffers->Title);
} if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { return RC_ERROR_FILE; }
} fwrite(&Index->Header, sizeof(index_header), 1, Index->Metadata.Handle);
else if(IndexEntryInsertionStart >= 0)
if(!(Index->File.Handle = fopen(Index->File.Path, "w"))) { return RC_ERROR_FILE; }
if(IndexEntryInsertionStart >= 0)
{ {
// Insert new // Insert new
fwrite(Index->Metadata.Buffer.Location + sizeof(index_header), sizeof(index_metadata) * N->ThisIndex, 1, Index->Metadata.Handle); fwrite(Index->Metadata.Buffer.Location + sizeof(index_header), sizeof(index_metadata) * N->ThisIndex, 1, Index->Metadata.Handle);
@ -5037,24 +4992,21 @@ InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, templ
fwrite(CollationBuffers->Search.Location, N->This.Size, 1, Index->File.Handle); fwrite(CollationBuffers->Search.Location, N->This.Size, 1, Index->File.Handle);
fwrite(Index->File.Buffer.Location + IndexEntryInsertionStart, Index->File.FileSize - IndexEntryInsertionStart, 1, Index->File.Handle); fwrite(Index->File.Buffer.Location + IndexEntryInsertionStart, Index->File.FileSize - IndexEntryInsertionStart, 1, Index->File.Handle);
if(VideoIsPrivate) if(!VideoIsPrivate)
{ {
if(!RecheckingPrivacy) LogError(LOG_NOTICE, "Inserted %s - %s", BaseFilename, CollationBuffers->Title);
fprintf(stderr, "\e[1;32mInserted\e[0m %s - %s\n", BaseFilename, CollationBuffers->Title);
}
else if(!RecheckingPrivacy)
{ {
LogError(LOG_NOTICE, "Privately Inserted %s", BaseFilename); LogError(LOG_NOTICE, "Privately Inserted %s", BaseFilename);
fprintf(stderr, "\e[0;34mPrivately Inserted\e[0m %s\n", BaseFilename); fprintf(stderr, "\e[0;34mPrivately Inserted\e[0m %s\n", BaseFilename);
} }
} }
else else
{
LogError(LOG_NOTICE, "Inserted %s - %s", BaseFilename, CollationBuffers->Title);
fprintf(stderr, "\e[1;32mInserted\e[0m %s - %s\n", BaseFilename, CollationBuffers->Title);
}
}
else
{ {
// Append new // Append new
if(IndexMetadataFileReadCode == RC_SUCCESS) if(Index->Metadata.FileSize > 0)
{ {
fwrite(Index->Metadata.Buffer.Location + sizeof(index_header), Index->Metadata.FileSize - sizeof(index_header), 1, Index->Metadata.Handle); fwrite(Index->Metadata.Buffer.Location + sizeof(index_header), Index->Metadata.FileSize - sizeof(index_header), 1, Index->Metadata.Handle);
fwrite(Index->File.Buffer.Location, Index->File.FileSize, 1, Index->File.Handle); fwrite(Index->File.Buffer.Location, Index->File.FileSize, 1, Index->File.Handle);
@ -5065,28 +5017,30 @@ InsertIntoIndex(index *Index, neighbourhood *N, buffers *CollationBuffers, templ
} }
fwrite(&N->This, sizeof(index_metadata), 1, Index->Metadata.Handle); fwrite(&N->This, sizeof(index_metadata), 1, Index->Metadata.Handle);
fwrite(CollationBuffers->Search.Location, N->This.Size, 1, Index->File.Handle); fwrite(CollationBuffers->Search.Location, N->This.Size, 1, Index->File.Handle);
if(VideoIsPrivate)
if(!VideoIsPrivate)
{ {
if(!RecheckingPrivacy) LogError(LOG_NOTICE, "Appended %s - %s", BaseFilename, CollationBuffers->Title);
fprintf(stderr, "\e[1;32mAppended\e[0m %s - %s\n", BaseFilename, CollationBuffers->Title);
}
else if(!RecheckingPrivacy)
{ {
LogError(LOG_NOTICE, "Privately Appended %s", BaseFilename); LogError(LOG_NOTICE, "Privately Appended %s", BaseFilename);
fprintf(stderr, "\e[0;34mPrivately Appended\e[0m %s\n", BaseFilename); fprintf(stderr, "\e[0;34mPrivately Appended\e[0m %s\n", BaseFilename);
} }
} }
else
{
LogError(LOG_NOTICE, "Appended %s - %s", BaseFilename, CollationBuffers->Title);
fprintf(stderr, "\e[1;32mAppended\e[0m %s - %s\n", BaseFilename, CollationBuffers->Title);
}
}
fclose(Index->Metadata.Handle); fclose(Index->Metadata.Handle);
fclose(Index->File.Handle);
FreeBuffer(&Index->Metadata.Buffer); FreeBuffer(&Index->Metadata.Buffer);
ReadFileIntoBuffer(&Index->Metadata, 0);
fclose(Index->File.Handle);
FreeBuffer(&Index->File.Buffer); FreeBuffer(&Index->File.Buffer);
ReadFileIntoBuffer(&Index->File, 0);
}
// TODO(matt): Remove VideoIsPrivate in favour of generating a player page in a random location // TODO(matt): Remove VideoIsPrivate in favour of generating a player page in a random location
return VideoIsPrivate ? RC_PRIVATE_VIDEO : Reinserting ? RC_SUCCESS : RC_UNFOUND; return VideoIsPrivate ? RC_PRIVATE_VIDEO : RC_SUCCESS;
} }
void void
@ -5279,19 +5233,6 @@ DeleteNeighbourLinks(file_buffer *File, index_metadata *Metadata)
int int
LinkNeighbours(index *Index, neighbourhood *N, char *BaseFilename, int LinkType) LinkNeighbours(index *Index, neighbourhood *N, char *BaseFilename, int LinkType)
{ {
switch(ReadFileIntoBuffer(&Index->Metadata, 0))
{
case RC_ERROR_FILE:
return RC_ERROR_FILE;
case RC_ERROR_MEMORY:
LogError(LOG_ERROR, "LinkNeighbours(): %s", strerror(errno));
return RC_ERROR_MEMORY;
case RC_SUCCESS:
break;
}
Index->Header = *(index_header *)Index->Metadata.Buffer.Ptr;
if(N->PrevIndex == -1 && N->NextIndex == -1) if(N->PrevIndex == -1 && N->NextIndex == -1)
{ {
buffer LonePlayerPagePath; buffer LonePlayerPagePath;
@ -5304,9 +5245,6 @@ LinkNeighbours(index *Index, neighbourhood *N, char *BaseFilename, int LinkType)
DeleteNeighbourLinks(&LonePlayerPage, &N->This); DeleteNeighbourLinks(&LonePlayerPage, &N->This);
DeclaimBuffer(&LonePlayerPagePath); DeclaimBuffer(&LonePlayerPagePath);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->ThisIndex;
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->This;
} }
else else
{ {
@ -5335,15 +5273,11 @@ LinkNeighbours(index *Index, neighbourhood *N, char *BaseFilename, int LinkType)
InsertNeighbourLink(&ThisPlayerPage, &N->This, &N->Prev, LINK_PREV, Index->Header.ProjectName, N->NextIndex == -1 ? TRUE : FALSE); InsertNeighbourLink(&ThisPlayerPage, &N->This, &N->Prev, LINK_PREV, Index->Header.ProjectName, N->NextIndex == -1 ? TRUE : FALSE);
DeclaimBuffer(&ThisPlayerPagePath); DeclaimBuffer(&ThisPlayerPagePath);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->ThisIndex;
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->This;
} }
case LINK_INCLUDE: case LINK_INCLUDE:
{ {
InsertNeighbourLink(&PreviousPlayerPage, &N->Prev, &N->This, LINK_NEXT, Index->Header.ProjectName, N->PrevIsFirst); InsertNeighbourLink(&PreviousPlayerPage, &N->Prev, &N->This, LINK_NEXT, Index->Header.ProjectName, N->PrevIsFirst);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->PrevIndex; *(index_metadata *)(Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->PrevIndex) = N->Prev;
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->Prev;
} }
} }
@ -5375,28 +5309,17 @@ LinkNeighbours(index *Index, neighbourhood *N, char *BaseFilename, int LinkType)
InsertNeighbourLink(&ThisPlayerPage, &N->This, &N->Next, LINK_NEXT, Index->Header.ProjectName, N->PrevIndex == -1 ? TRUE : FALSE); InsertNeighbourLink(&ThisPlayerPage, &N->This, &N->Next, LINK_NEXT, Index->Header.ProjectName, N->PrevIndex == -1 ? TRUE : FALSE);
DeclaimBuffer(&ThisPlayerPagePath); DeclaimBuffer(&ThisPlayerPagePath);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->ThisIndex;
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->This;
} }
case LINK_INCLUDE: case LINK_INCLUDE:
{ {
InsertNeighbourLink(&NextPlayerPage, &N->Next, &N->This, LINK_PREV, Index->Header.ProjectName, N->NextIsFinal); InsertNeighbourLink(&NextPlayerPage, &N->Next, &N->This, LINK_PREV, Index->Header.ProjectName, N->NextIsFinal);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->NextIndex; *(index_metadata *)(Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->NextIndex) = N->Next;
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->Next;
} }
} }
DeclaimBuffer(&NextPlayerPagePath); DeclaimBuffer(&NextPlayerPagePath);
} }
} }
Index->Metadata.Handle = fopen(Index->Metadata.Path, "w");
fwrite(Index->Metadata.Buffer.Location, Index->Metadata.FileSize, 1, Index->Metadata.Handle);
fclose(Index->Metadata.Handle);
FreeBuffer(&Index->Metadata.Buffer);
return RC_SUCCESS; return RC_SUCCESS;
} }
@ -5457,30 +5380,8 @@ int
DeleteFromIndex(index *Index, neighbourhood *N, char *BaseFilename) DeleteFromIndex(index *Index, neighbourhood *N, char *BaseFilename)
{ {
// TODO(matt): LogError() // TODO(matt): LogError()
switch(ReadFileIntoBuffer(&Index->Metadata, 0)) Index->Header = *(index_header *)Index->Metadata.Buffer.Location;
{ Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(Index->Header);
case RC_ERROR_FILE:
return RC_ERROR_FILE;
case RC_ERROR_MEMORY:
LogError(LOG_ERROR, "DeleteFromIndex(): %s", strerror(errno));
return RC_ERROR_MEMORY;
case RC_SUCCESS:
break;
}
switch(ReadFileIntoBuffer(&Index->File, 0))
{
case RC_ERROR_FILE:
return RC_ERROR_FILE;
case RC_ERROR_MEMORY:
LogError(LOG_ERROR, "DeleteFromIndex(): %s", strerror(errno));
return RC_ERROR_MEMORY;
case RC_SUCCESS:
break;
}
Index->Header = *(index_header *)Index->Metadata.Buffer.Ptr;
Index->Metadata.Buffer.Ptr += sizeof(Index->Header);
index_metadata *Entry = { 0 }; index_metadata *Entry = { 0 };
int EntryIndex = BinarySearchForMetadataEntry(Index, &Entry, BaseFilename); int EntryIndex = BinarySearchForMetadataEntry(Index, &Entry, BaseFilename);
@ -5496,8 +5397,14 @@ DeleteFromIndex(index *Index, neighbourhood *N, char *BaseFilename)
if(Index->Header.EntryCount == 0) if(Index->Header.EntryCount == 0)
{ {
DeleteIndexPageFromFilesystem(); DeleteIndexPageFromFilesystem();
remove(Index->Metadata.Path); remove(Index->Metadata.Path);
Index->Metadata.FileSize = 0;
FreeBuffer(&Index->Metadata.Buffer);
remove(Index->File.Path); remove(Index->File.Path);
Index->File.FileSize = 0;
FreeBuffer(&Index->File.Buffer);
} }
else else
{ {
@ -5511,24 +5418,27 @@ DeleteFromIndex(index *Index, neighbourhood *N, char *BaseFilename)
1, Index->Metadata.Handle); 1, Index->Metadata.Handle);
fclose(Index->Metadata.Handle); fclose(Index->Metadata.Handle);
FreeBuffer(&Index->Metadata.Buffer);
ReadFileIntoBuffer(&Index->Metadata, 0);
fwrite(Index->File.Buffer.Location, DeleteFileFrom, 1, Index->File.Handle); fwrite(Index->File.Buffer.Location, DeleteFileFrom, 1, Index->File.Handle);
fwrite(Index->File.Buffer.Location + DeleteFileTo, Index->File.FileSize - DeleteFileTo, 1, Index->File.Handle); fwrite(Index->File.Buffer.Location + DeleteFileTo, Index->File.FileSize - DeleteFileTo, 1, Index->File.Handle);
fclose(Index->File.Handle); fclose(Index->File.Handle);
FreeBuffer(&Index->File.Buffer);
ReadFileIntoBuffer(&Index->File, 0);
} }
} }
FreeBuffer(&Index->Metadata.Buffer);
FreeBuffer(&Index->File.Buffer);
return Entry ? RC_SUCCESS : RC_NOOP; return Entry ? RC_SUCCESS : RC_NOOP;
} }
int int
IndexToBuffer(index *Index, buffers *CollationBuffers) // NOTE(matt): This guy malloc's CollationBuffers->Index IndexToBuffer(index *Index, buffers *CollationBuffers) // NOTE(matt): This guy malloc's CollationBuffers->Index
{ {
if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_SUCCESS) if(Index->Metadata.FileSize > 0)
{ {
Index->Header = *(index_header*)Index->Metadata.Buffer.Ptr; Index->Header = *(index_header*)Index->Metadata.Buffer.Location;
Index->Metadata.Buffer.Ptr += sizeof(Index->Header);
bool ProjectFound = FALSE; bool ProjectFound = FALSE;
int ProjectIndex; int ProjectIndex;
@ -5544,7 +5454,6 @@ IndexToBuffer(index *Index, buffers *CollationBuffers) // NOTE(matt): This guy m
if(!ProjectFound) if(!ProjectFound)
{ {
fprintf(stderr, "Missing Project Info for %s\n", Config.ProjectID); fprintf(stderr, "Missing Project Info for %s\n", Config.ProjectID);
FreeBuffer(&Index->Metadata.Buffer);
return RC_ERROR_PROJECT; return RC_ERROR_PROJECT;
} }
@ -5599,11 +5508,8 @@ IndexToBuffer(index *Index, buffers *CollationBuffers) // NOTE(matt): This guy m
int EntryLength = StringLength(PlayerURL.Location) + sizeof(Text) + 82; int EntryLength = StringLength(PlayerURL.Location) + sizeof(Text) + 82;
CollationBuffers->Index.Size = StringLength(queryContainer) + (Index->Header.EntryCount * EntryLength) + StringLength(Script); CollationBuffers->Index.Size = StringLength(queryContainer) + (Index->Header.EntryCount * EntryLength) + StringLength(Script);
if(!(CollationBuffers->Index.Location = malloc(CollationBuffers->Index.Size))) if(!(CollationBuffers->Index.Location = malloc(CollationBuffers->Index.Size))) { return RC_ERROR_MEMORY; }
{
FreeBuffer(&Index->Metadata.Buffer);
return(RC_ERROR_MEMORY);
}
CollationBuffers->Index.ID = "Index"; CollationBuffers->Index.ID = "Index";
CollationBuffers->Index.Ptr = CollationBuffers->Index.Location; CollationBuffers->Index.Ptr = CollationBuffers->Index.Location;
@ -5612,6 +5518,7 @@ IndexToBuffer(index *Index, buffers *CollationBuffers) // NOTE(matt): This guy m
int ProjectIDLength = StringLength(Config.ProjectID); int ProjectIDLength = StringLength(Config.ProjectID);
bool IndexRequired = FALSE; bool IndexRequired = FALSE;
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(Index->Header);
for(int EntryIndex = 0; EntryIndex < Index->Header.EntryCount; ++EntryIndex, Index->Metadata.Buffer.Ptr += sizeof(index_metadata)) for(int EntryIndex = 0; EntryIndex < Index->Header.EntryCount; ++EntryIndex, Index->Metadata.Buffer.Ptr += sizeof(index_metadata))
{ {
This = (index_metadata *)Index->Metadata.Buffer.Ptr; This = (index_metadata *)Index->Metadata.Buffer.Ptr;
@ -5667,7 +5574,6 @@ IndexToBuffer(index *Index, buffers *CollationBuffers) // NOTE(matt): This guy m
CopyStringToBuffer(&CollationBuffers->Index, "%s", Script); CopyStringToBuffer(&CollationBuffers->Index, "%s", Script);
FreeBuffer(&Index->Metadata.Buffer);
if(!IndexRequired) { return RC_NOOP; } if(!IndexRequired) { return RC_NOOP; }
else { return RC_SUCCESS; } else { return RC_SUCCESS; }
} }
@ -5742,16 +5648,14 @@ GeneratePlayerPage(index *Index, neighbourhood *N, buffers *CollationBuffers, te
break; break;
} }
} }
BuffersToHTML(CollationBuffers, PlayerTemplate, PlayerPagePath, PAGE_PLAYER, &N->This.LinkOffsets.PrevStart); BuffersToHTML(CollationBuffers, PlayerTemplate, PlayerPagePath, PAGE_PLAYER, &N->This.LinkOffsets.PrevStart);
ReadFileIntoBuffer(&Index->Metadata, 0); *(index_metadata *)(Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->ThisIndex) = N->This;
if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { FreeBuffer(&Index->Metadata.Buffer); return RC_ERROR_FILE; } Index->Metadata.Handle = fopen(Index->Metadata.Path, "w");
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + sizeof(index_header) + sizeof(index_metadata) * N->ThisIndex; fwrite(Index->Metadata.Buffer.Location, Index->Metadata.Buffer.Size, 1, Index->Metadata.Handle);
*(index_metadata *)Index->Metadata.Buffer.Ptr = N->This;
fwrite(Index->Metadata.Buffer.Location, Index->Metadata.FileSize, 1, Index->Metadata.Handle);
fclose(Index->Metadata.Handle); fclose(Index->Metadata.Handle);
FreeBuffer(&Index->Metadata.Buffer);
if(IndexInTemplate) if(IndexInTemplate)
{ {
@ -5814,16 +5718,12 @@ DeleteEntry(index *Index, neighbourhood *Neighbourhood, char *BaseFilename)
return RC_NOOP; return RC_NOOP;
} }
void int
InsertEntry(index *Index, neighbourhood *Neighbourhood, buffers *CollationBuffers, template *PlayerTemplate, template *BespokeTemplate, char *BaseFilename, bool *Inserted, bool RecheckingPrivacy) InsertEntry(index *Index, neighbourhood *Neighbourhood, buffers *CollationBuffers, template *PlayerTemplate, template *BespokeTemplate, char *BaseFilename, bool RecheckingPrivacy)
{ {
switch(InsertIntoIndex(Index, Neighbourhood, CollationBuffers, &BespokeTemplate, BaseFilename, RecheckingPrivacy)) if(InsertIntoIndex(Index, Neighbourhood, CollationBuffers, &BespokeTemplate, BaseFilename, RecheckingPrivacy) == RC_SUCCESS)
{ {
case RC_UNFOUND:
LinkNeighbours(Index, Neighbourhood, BaseFilename, LINK_INCLUDE); LinkNeighbours(Index, Neighbourhood, BaseFilename, LINK_INCLUDE);
*Inserted = TRUE;
case RC_SUCCESS:
{
if(StringsDiffer(BespokeTemplate->Metadata.Filename, "")) if(StringsDiffer(BespokeTemplate->Metadata.Filename, ""))
{ {
GeneratePlayerPage(Index, Neighbourhood, CollationBuffers, BespokeTemplate, BaseFilename); GeneratePlayerPage(Index, Neighbourhood, CollationBuffers, BespokeTemplate, BaseFilename);
@ -5833,17 +5733,17 @@ InsertEntry(index *Index, neighbourhood *Neighbourhood, buffers *CollationBuffer
{ {
GeneratePlayerPage(Index, Neighbourhood, CollationBuffers, PlayerTemplate, BaseFilename); GeneratePlayerPage(Index, Neighbourhood, CollationBuffers, PlayerTemplate, BaseFilename);
} }
*Inserted = TRUE; return RC_SUCCESS;
} break;
} }
return RC_NOOP;
} }
int int
RecheckPrivacy(index *Index, buffers *CollationBuffers, template *IndexTemplate, template *PlayerTemplate, template *BespokeTemplate) RecheckPrivacy(index *Index, buffers *CollationBuffers, template *IndexTemplate, template *PlayerTemplate, template *BespokeTemplate)
{ {
if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_SUCCESS) if(Index->Metadata.FileSize > 0)
{ {
Index->Header = *(index_header*)Index->Metadata.Buffer.Ptr; Index->Header = *(index_header*)Index->Metadata.Buffer.Location;
index_metadata Entry = { }; index_metadata Entry = { };
int PrivateEntryIndex = 0; int PrivateEntryIndex = 0;
index_metadata PrivateEntries[Index->Header.EntryCount]; index_metadata PrivateEntries[Index->Header.EntryCount];
@ -5857,7 +5757,6 @@ RecheckPrivacy(index *Index, buffers *CollationBuffers, template *IndexTemplate,
++PrivateEntryIndex; ++PrivateEntryIndex;
} }
} }
FreeBuffer(&Index->Metadata.Buffer);
for(int i = 0; i < PrivateEntryIndex; ++i) for(int i = 0; i < PrivateEntryIndex; ++i)
{ {
@ -5865,7 +5764,7 @@ RecheckPrivacy(index *Index, buffers *CollationBuffers, template *IndexTemplate,
Neighbourhood.PrevIndex = -1; Neighbourhood.PrevIndex = -1;
Neighbourhood.ThisIndex = -1; Neighbourhood.ThisIndex = -1;
Neighbourhood.NextIndex = -1; Neighbourhood.NextIndex = -1;
InsertEntry(Index, &Neighbourhood, CollationBuffers, PlayerTemplate, BespokeTemplate, PrivateEntries[i].BaseFilename, &Inserted, TRUE); Inserted = (InsertEntry(Index, &Neighbourhood, CollationBuffers, PlayerTemplate, BespokeTemplate, PrivateEntries[i].BaseFilename, TRUE) == RC_SUCCESS);
} }
if(Inserted) if(Inserted)
@ -5895,9 +5794,10 @@ MonitorDirectory(index *Index, buffers *CollationBuffers, template *IndexTemplat
int BytesRead = read(inotifyInstance, Events.Location, Events.Size); // TODO(matt): Handle error EINVAL int BytesRead = read(inotifyInstance, Events.Location, Events.Size); // TODO(matt): Handle error EINVAL
if(inotifyInstance < 0) { perror("MonitorDirectory()"); } if(inotifyInstance < 0) { perror("MonitorDirectory()"); }
bool Deleted = FALSE; struct inotify_event *FinalFileEvents[1024];
bool Inserted = FALSE; int FinalFileEventsCount = 0;
// TODO(matt): Test this with longer update intervals, and combinations of events...
for(Events.Ptr = Events.Location; for(Events.Ptr = Events.Location;
Events.Ptr < Events.Location + BytesRead && Events.Ptr - Events.Location < Events.Size; Events.Ptr < Events.Location + BytesRead && Events.Ptr - Events.Location < Events.Size;
Events.Ptr += sizeof(struct inotify_event) + Event->len) Events.Ptr += sizeof(struct inotify_event) + Event->len)
@ -5908,23 +5808,37 @@ MonitorDirectory(index *Index, buffers *CollationBuffers, template *IndexTemplat
if(!(StringsDiffer(Ptr, ".hmml"))) if(!(StringsDiffer(Ptr, ".hmml")))
{ {
*Ptr = '\0'; *Ptr = '\0';
bool FoundEvent = FALSE;
int FinalFileEventsIndex;
for(FinalFileEventsIndex = 0; FinalFileEventsIndex < FinalFileEventsCount; ++FinalFileEventsIndex)
{
if(!StringsDiffer(FinalFileEvents[FinalFileEventsIndex]->name, Event->name))
{
FinalFileEvents[FinalFileEventsIndex] = Event;
FoundEvent = TRUE;
}
}
if(!FoundEvent) { FinalFileEvents[FinalFileEventsIndex] = Event; ++FinalFileEventsCount; }
}
}
bool Deleted = FALSE;
bool Inserted = FALSE;
for(int FinalFileEventsIndex = 0; FinalFileEventsIndex < FinalFileEventsCount; ++FinalFileEventsIndex)
{
neighbourhood Neighbourhood = { }; neighbourhood Neighbourhood = { };
Neighbourhood.PrevIndex = -1; Neighbourhood.PrevIndex = -1;
Neighbourhood.ThisIndex = -1; Neighbourhood.ThisIndex = -1;
Neighbourhood.NextIndex = -1; Neighbourhood.NextIndex = -1;
// TODO(matt): Maybe handle IN_ALL_EVENTS // TODO(matt): Maybe handle IN_ALL_EVENTS
if(Event->mask & IN_DELETE || Event->mask & IN_MOVED_FROM) if(FinalFileEvents[FinalFileEventsIndex]->mask & IN_DELETE || FinalFileEvents[FinalFileEventsIndex]->mask & IN_MOVED_FROM)
{ {
if(DeleteEntry(Index, &Neighbourhood, Event->name) == RC_SUCCESS) Deleted = (DeleteEntry(Index, &Neighbourhood, FinalFileEvents[FinalFileEventsIndex]->name) == RC_SUCCESS);
{
Deleted = TRUE;
}
} }
else else
{ {
InsertEntry(Index, &Neighbourhood, CollationBuffers, PlayerTemplate, BespokeTemplate, Event->name, &Inserted, 0); Inserted = (InsertEntry(Index, &Neighbourhood, CollationBuffers, PlayerTemplate, BespokeTemplate, FinalFileEvents[FinalFileEventsIndex]->name, 0) == RC_SUCCESS);
}
} }
} }
@ -6057,17 +5971,6 @@ UpgradeDB(index *Index)
fwrite(&Index->Header, sizeof(Index->Header), 1, Index->Metadata.Handle); fwrite(&Index->Header, sizeof(Index->Header), 1, Index->Metadata.Handle);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + OriginalHeaderSize; Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + OriginalHeaderSize;
if(ReadFileIntoBuffer(&Index->File, 0) == RC_ERROR_FILE)
{
fprintf(stderr, "\e[1;31mUnable to open index file\e[0m %s: %s\n"
"Removing %s and starting afresh\n", Index->File.Path, strerror(errno),
Index->Metadata.Path);
fclose(Index->Metadata.Handle);
FreeBuffer(&Index->Metadata.Buffer);
remove(Index->Metadata.Path);
return RC_ERROR_FILE;
}
Index->File.Buffer.Ptr += StringLength("---\n"); Index->File.Buffer.Ptr += StringLength("---\n");
for(int EntryIndex = 0; EntryIndex < Index->Header.EntryCount; ++EntryIndex) for(int EntryIndex = 0; EntryIndex < Index->Header.EntryCount; ++EntryIndex)
@ -6097,8 +6000,6 @@ UpgradeDB(index *Index)
Index->File.Buffer.Ptr = IndexEntryStart; Index->File.Buffer.Ptr = IndexEntryStart;
} }
FreeBuffer(&Index->File.Buffer);
fclose(Index->Metadata.Handle); fclose(Index->Metadata.Handle);
FreeBuffer(&Index->Metadata.Buffer); FreeBuffer(&Index->Metadata.Buffer);
if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_ERROR_FILE) if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_ERROR_FILE)
@ -6112,9 +6013,7 @@ UpgradeDB(index *Index)
{ {
if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { FreeBuffer(&Index->Metadata.Buffer); return RC_ERROR_FILE; } if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { FreeBuffer(&Index->Metadata.Buffer); return RC_ERROR_FILE; }
fwrite(&Index->Header, sizeof(Index->Header), 1, Index->Metadata.Handle); fwrite(&Index->Header, sizeof(Index->Header), 1, Index->Metadata.Handle);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location + OriginalHeaderSize; fwrite(Index->Metadata.Buffer.Location + OriginalHeaderSize, Index->Metadata.FileSize - OriginalHeaderSize, 1, Index->Metadata.Handle);
fwrite(Index->Metadata.Buffer.Ptr, Index->Metadata.FileSize - OriginalHeaderSize, 1, Index->Metadata.Handle);
Index->Metadata.Buffer.Ptr = Index->Metadata.Buffer.Location;
fclose(Index->Metadata.Handle); fclose(Index->Metadata.Handle);
FreeBuffer(&Index->Metadata.Buffer); FreeBuffer(&Index->Metadata.Buffer);
if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_ERROR_FILE) if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_ERROR_FILE)
@ -6154,12 +6053,7 @@ DeleteDeadIndexEntries(index *Index)
{ {
// TODO(matt): More rigorously figure out who we should delete // TODO(matt): More rigorously figure out who we should delete
// Maybe compare the output directory and the input HMML names // Maybe compare the output directory and the input HMML names
if(ReadFileIntoBuffer(&Index->Metadata, 0) == RC_ERROR_FILE) Index->Header = *(index_header *)Index->Metadata.Buffer.Location;
{
return RC_ERROR_FILE;
}
Index->Header = *(index_header *)Index->Metadata.Buffer.Ptr;
if(Index->Header.CurrentDBVersion < CINERA_DB_VERSION) if(Index->Header.CurrentDBVersion < CINERA_DB_VERSION)
{ {
if(CINERA_DB_VERSION == 4) if(CINERA_DB_VERSION == 4)
@ -6167,8 +6061,7 @@ DeleteDeadIndexEntries(index *Index)
fprintf(stderr, "\n\e[1;31mHandle conversion from CINERA_DB_VERSION %d to %d!\e[0m\n\n", Index->Header.CurrentDBVersion, CINERA_DB_VERSION); fprintf(stderr, "\n\e[1;31mHandle conversion from CINERA_DB_VERSION %d to %d!\e[0m\n\n", Index->Header.CurrentDBVersion, CINERA_DB_VERSION);
exit(RC_ERROR_FATAL); exit(RC_ERROR_FATAL);
} }
if(UpgradeDB(Index) == RC_ERROR_FILE) { return RC_NOOP; if(UpgradeDB(Index) == RC_ERROR_FILE) { return RC_NOOP; }
}
} }
else if(Index->Header.CurrentDBVersion > CINERA_DB_VERSION) else if(Index->Header.CurrentDBVersion > CINERA_DB_VERSION)
{ {
@ -6209,6 +6102,7 @@ DeleteDeadIndexEntries(index *Index)
DeclaimBuffer(&OldPlayerDirectory); DeclaimBuffer(&OldPlayerDirectory);
ClearCopyStringNoFormat(Index->Header.PlayerLocation, sizeof(Index->Header.PlayerLocation), Config.PlayerLocation); ClearCopyStringNoFormat(Index->Header.PlayerLocation, sizeof(Index->Header.PlayerLocation), Config.PlayerLocation);
*(index_header *)Index->Metadata.Buffer.Location = Index->Header;
NewPlayerLocation = TRUE; NewPlayerLocation = TRUE;
} }
if(StringsDiffer(Index->Header.IndexLocation, Config.IndexLocation)) if(StringsDiffer(Index->Header.IndexLocation, Config.IndexLocation))
@ -6234,13 +6128,13 @@ DeleteDeadIndexEntries(index *Index)
DeclaimBuffer(&OldIndexDirectory); DeclaimBuffer(&OldIndexDirectory);
ClearCopyStringNoFormat(Index->Header.IndexLocation, sizeof(Index->Header.IndexLocation), Config.IndexLocation); ClearCopyStringNoFormat(Index->Header.IndexLocation, sizeof(Index->Header.IndexLocation), Config.IndexLocation);
*(index_header *)Index->Metadata.Buffer.Location = Index->Header;
NewIndexLocation = TRUE; NewIndexLocation = TRUE;
} }
if(NewPlayerLocation || NewIndexLocation) if(NewPlayerLocation || NewIndexLocation)
{ {
if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { FreeBuffer(&Index->Metadata.Buffer); return RC_ERROR_FILE; } if(!(Index->Metadata.Handle = fopen(Index->Metadata.Path, "w"))) { FreeBuffer(&Index->Metadata.Buffer); return RC_ERROR_FILE; }
*(index_header *)Index->Metadata.Buffer.Location = Index->Header;
fwrite(Index->Metadata.Buffer.Location, Index->Metadata.FileSize, 1, Index->Metadata.Handle); fwrite(Index->Metadata.Buffer.Location, Index->Metadata.FileSize, 1, Index->Metadata.Handle);
fclose(Index->Metadata.Handle); fclose(Index->Metadata.Handle);
} }
@ -6298,7 +6192,6 @@ DeleteDeadIndexEntries(index *Index)
} }
} }
FreeBuffer(&Index->Metadata.Buffer);
return Deleted ? RC_SUCCESS : RC_NOOP; return Deleted ? RC_SUCCESS : RC_NOOP;
} }
@ -6306,7 +6199,7 @@ int
SyncIndexWithInput(index *Index, buffers *CollationBuffers, template *IndexTemplate, template *PlayerTemplate, template *BespokeTemplate) SyncIndexWithInput(index *Index, buffers *CollationBuffers, template *IndexTemplate, template *PlayerTemplate, template *BespokeTemplate)
{ {
bool Deleted = FALSE; bool Deleted = FALSE;
if(DeleteDeadIndexEntries(Index) == RC_SUCCESS) if(Index->Metadata.FileSize > 0 && Index->File.FileSize > 0 && DeleteDeadIndexEntries(Index) == RC_SUCCESS)
{ {
Deleted = TRUE; Deleted = TRUE;
} }
@ -6332,7 +6225,7 @@ SyncIndexWithInput(index *Index, buffers *CollationBuffers, template *IndexTempl
neighbourhood Neighbourhood = { }; neighbourhood Neighbourhood = { };
Neighbourhood.PrevIndex = -1; Neighbourhood.PrevIndex = -1;
Neighbourhood.NextIndex = -1; Neighbourhood.NextIndex = -1;
InsertEntry(Index, &Neighbourhood, CollationBuffers, PlayerTemplate, BespokeTemplate, ProjectFiles->d_name, &Inserted, 0); Inserted = (InsertEntry(Index, &Neighbourhood, CollationBuffers, PlayerTemplate, BespokeTemplate, ProjectFiles->d_name, 0) == RC_SUCCESS);
} }
} }
closedir(ProjectDirHandle); closedir(ProjectDirHandle);
@ -6738,19 +6631,23 @@ main(int ArgC, char **Args)
} }
index Index = { }; index Index = { };
Index.Metadata.Buffer.ID = "IndexMetadata"; Index.Metadata.Buffer.ID = "IndexMetadata";
CopyString(Index.Metadata.Path, sizeof(Index.Metadata.Path), "%s/%s.metadata", Config.BaseDir, Config.ProjectID); CopyString(Index.Metadata.Path, sizeof(Index.Metadata.Path), "%s/%s.metadata", Config.BaseDir, Config.ProjectID);
ReadFileIntoBuffer(&Index.Metadata, 0); // NOTE(matt): Could we actually catch errors (permissions?) here and bail?
Index.File.Buffer.ID = "IndexFile"; Index.File.Buffer.ID = "IndexFile";
CopyString(Index.File.Path, sizeof(Index.File.Path), "%s/%s.index", Config.BaseDir, Config.ProjectID); CopyString(Index.File.Path, sizeof(Index.File.Path), "%s/%s.index", Config.BaseDir, Config.ProjectID);
ReadFileIntoBuffer(&Index.File, 0); // NOTE(matt): Could we actually catch errors (permissions?) here and bail?
printf("┌╼ Synchronising with annotation files in Project Input Directory ╾┐\n"); printf("┌╼ Synchronising with Annotations Directory ╾┐\n");
SyncIndexWithInput(&Index, &CollationBuffers, IndexTemplate, PlayerTemplate, BespokeTemplate); SyncIndexWithInput(&Index, &CollationBuffers, IndexTemplate, PlayerTemplate, BespokeTemplate);
if(Config.Mode & MODE_ONESHOT) if(Config.Mode & MODE_ONESHOT)
{ {
goto RIP; goto RIP;
} }
printf("\n┌╼ Monitoring Project Directory for \e[1;32mnew\e[0m, \e[1;33medited\e[0m and \e[1;30mdeleted\e[0m .hmml files ╾┐\n"); printf("\n┌╼ Monitoring Annotations Directory for \e[1;32mnew\e[0m, \e[1;33medited\e[0m and \e[1;30mdeleted\e[0m .hmml files ╾┐\n");
int inotifyInstance = inotify_init1(IN_NONBLOCK); int inotifyInstance = inotify_init1(IN_NONBLOCK);
// NOTE(matt): Do we want to also watch IN_DELETE_SELF events? // NOTE(matt): Do we want to also watch IN_DELETE_SELF events?
int WatchDescriptor = inotify_add_watch(inotifyInstance, Config.ProjectDir, IN_CLOSE_WRITE | IN_DELETE | IN_MOVED_FROM | IN_MOVED_TO); int WatchDescriptor = inotify_add_watch(inotifyInstance, Config.ProjectDir, IN_CLOSE_WRITE | IN_DELETE | IN_MOVED_FROM | IN_MOVED_TO);
@ -6795,7 +6692,7 @@ NextFile:
CopyString(Config.SingleHMMLFilePath, sizeof(Config.SingleHMMLFilePath), "%s", Args[FileIndex]); CopyString(Config.SingleHMMLFilePath, sizeof(Config.SingleHMMLFilePath), "%s", Args[FileIndex]);
switch(HMMLToBuffers(&CollationBuffers, &BespokeTemplate, Args[FileIndex], 0)) switch(HMMLToBuffers(&CollationBuffers, &BespokeTemplate, Args[FileIndex], 0))
{ {
// TODO(matt): Actually sort out the fatality of these cases, once we are always-on // TODO(matt): Actually sort out the fatality of these cases
case RC_ERROR_FILE: case RC_ERROR_FILE:
case RC_ERROR_FATAL: case RC_ERROR_FATAL:
goto RIP; goto RIP;
@ -6816,7 +6713,7 @@ NextFile:
0, 0,
PAGE_PLAYER, 0)) PAGE_PLAYER, 0))
{ {
// TODO(matt): Actually sort out the fatality of these cases, once we are always-on // TODO(matt): Actually sort out the fatality of these cases
case RC_INVALID_TEMPLATE: case RC_INVALID_TEMPLATE:
if(HasBespokeTemplate) { DeclaimTemplate(BespokeTemplate); } if(HasBespokeTemplate) { DeclaimTemplate(BespokeTemplate); }
if(FileIndex < (ArgC - 1)) { goto NextFile; } if(FileIndex < (ArgC - 1)) { goto NextFile; }

2
cinera/cinera_search.js
View File

@ -233,7 +233,7 @@ xhr.addEventListener("load", function() {
} else if (line.startsWith("name:")) { } else if (line.startsWith("name:")) {
episode.name = line.slice(6); episode.name = line.slice(6);
} else if (line.startsWith("title:")) { } else if (line.startsWith("title:")) {
episode.title = line.slice(7).replace(/"/g, ""); episode.title = line.slice(7).trim().slice(1, -1);
} else if (line.startsWith("markers")) { } else if (line.startsWith("markers")) {
mode = "markers"; mode = "markers";
episode.markers = []; episode.markers = [];