mirror of
				https://github.com/KevinMidboe/linguist.git
				synced 2025-10-29 17:50:22 +00:00 
			
		
		
		
	Handle case where newline chars don't transcode to detected encoding
We've seen cases where binary files are detected as encodings such as ISO-8859-8-I. This usually happens when the binary files are short, so while the detector is mistaken, there is also not very much data for use in the detection algorithm in the first place so it's understandable that the detector was wrong. In these cases, the code to convert ASCII newline characters to encodings such as ISO-8859-8-I fails because there is no conversion between them. We now simply assume that the data is all one line in those cases. In reality the data is binary, but this obviously difficult to detect reliably.
This commit is contained in:
		
							
								
								
									
										1
									
								
								samples/Text/iso8859-8-i.txt
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										1
									
								
								samples/Text/iso8859-8-i.txt
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1 @@ | ||||
| %<25><><EFBFBD> | ||||
		Reference in New Issue
	
	Block a user