Merge.txt
But it would be better and easier if I could just call one batch file from the main which would perform the same task for each subfolder. I tried for /r %%d in (.) do (copy *.txt merge.txt /B) but without success. The error message is that *.txt can't be found which leads me to believe I can't just use the wildcard but rather would need to specify the files. But those are randomly generated.
Merge.txt
Could you give me hint to fix this problem?My ultimate goal is to train Bart model with my language.Or is it okay that tokenizer encode into 3 tokens?Or can I modify vocab.json and merge.txt manually to let BartTokenizer encode into [0] ?
For example, this line in .gitattributes can be used to tell the mergemachinery to leave much longer (instead of the usual 7-character-long)conflict markers when merging the file Documentation/git-merge.txtresults in a conflict. 041b061a72