Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ファイルの文字コード検出処理を変更する #54

Open
sonoh5n opened this issue Oct 7, 2024 · 2 comments
Open

ファイルの文字コード検出処理を変更する #54

sonoh5n opened this issue Oct 7, 2024 · 2 comments
Assignees
@sonoh5n
Labels
🐛 bug Something isn't working 🛠️ improvement Improve features which are working 🐎 enhancement Overall performance and improvement ♻️ refactor refactoring
Milestone
Version 1.1.0

Comments

@sonoh5n
Copy link
Collaborator

sonoh5n commented Oct 7, 2024

rdetoolkit/rde2util.py::CharDecEncoding.detect_text_file_encodingの処理を変更します。
@sonoh5n sonoh5n self-assigned this Oct 7, 2024
@sonoh5n sonoh5n added this to the Version 1.0.3 milestone Oct 30, 2024
@sonoh5n sonoh5n added 🐛 bug Something isn't working 🐎 enhancement Overall performance and improvement 🛠️ improvement Improve features which are working ♻️ refactor refactoring labels Oct 30, 2024
@sonoh5n
Copy link
Collaborator Author

sonoh5n commented Dec 16, 2024

chardetngを利用することで上記の問題を解決できる。ネガティブマッチング->ポジティブマッチングで検出する。

@sonoh5n
Copy link
Collaborator Author

sonoh5n commented Dec 23, 2024

ISO8859-2, Windows1252, ISO8859-2でテスト

sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 add charset detection and file reading functions
6a12161
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 add image resizing and file handling functions
4fe5994
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 deprecate read_from_json_file and write_to_json_file functi…
c42a5b5 
…ons, use fileops instead
@sonoh5n sonoh5n mentioned this issue Dec 23, 2024
Merged
2 tasks
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 add documentation for file operations in rdetoolkit
d27ffb9
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 add fileops module to documentation for RDE-related file op…
47d9463 
…erations
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 improve error handling in charset_detector and add comprehe…
01a21c8 
…nsive tests for encoding detection
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 Improve error handling in encoding detection functions
2847d89
sonoh5n added a commit that referenced this issue Dec 23, 2024
@sonoh5n
feat: #54 Update error handling in encoding detection to use PyValueE…
7f274e1 
…rror for empty files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sonoh5n sonoh5n

Labels
🐛 bug Something isn't working 🛠️ improvement Improve features which are working 🐎 enhancement Overall performance and improvement ♻️ refactor refactoring
Projects
None yet
Milestone
Version 1.1.0
Development

No branches or pull requests
1 participant
@sonoh5n