Merge 1.1.377

docs/mypy-comparison.md

@@ -9,7 +9,9 @@ Mypy is the “OG” in the world of Python type checkers. It was started by Juk
 
 Mypy served as a reference implementation of [PEP 484](https://www.python.org/dev/peps/pep-0484/), which defines standard behaviors for Python static typing. Although PEP 484 spells out many type checking behaviors, it intentionally leaves many other behaviors undefined. This approach has allowed different type checkers to innovate and differentiate.
 
-Pyright generally adheres to the type checking behaviors spelled out in PEP 484 and follow-on typing PEPs (526, 544, 586, 589, etc.). For behaviors that are not explicitly spelled out in these standards, pyright generally tries to adhere to mypy’s behavior unless there is a compelling justification for deviating. This document discusses these differences and provides the reasoning behind each design choice.
+Pyright generally adheres to the official [Python typing specification](https://typing.readthedocs.io/en/latest/spec/index.html), which incorporates and builds upon PEP 484 and other typing-related PEPs. The typing spec is accompanied by an ever-expanding suite of conformance tests. For the latest conformance test results for pyright, mypy and other type checkers, refer to [this page](https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html).
+
+For behaviors that are not explicitly spelled out in the typing spec, pyright generally tries to adhere to mypy’s behavior unless there is a compelling justification for deviating. This document discusses these differences and provides the reasoning behind each design choice.
 
 
 ### Design Goals
@@ -18,9 +20,9 @@ Pyright was designed with performance in mind. It is not unusual for pyright to
 
 Pyright was also designed to be used as the foundation for a Python [language server](https://microsoft.github.io/language-server-protocol/). Language servers provide interactive programming features such as completion suggestions, function signature help, type information on hover, semantic-aware search, semantic-aware renaming, semantic token coloring, refactoring tools, etc. For a good user experience, these features require highly responsive type evaluation performance during interactive code modification. They also require type evaluation to work on code that is incomplete and contains syntax errors.
 
-To achieve these design goals, pyright is implemented as a “lazy” or “just-in-time” type evaluator. Rather than analyzing all code in a module from top to bottom, it is able to evaluate the type of an arbitrary identifier anywhere within a module. If the type of that identifier depends on the types of other expressions or symbols, pyright recursively evaluates those in turn until it has enough information to determine the type of the requested identifier. By comparison, mypy uses a more traditional multi-pass architecture where semantic analysis is performed multiple times on a module from the top to the bottom until all types converge.
+To achieve these design goals, pyright is implemented as a “lazy” or “just-in-time” type evaluator. Rather than analyzing all code in a module from top to bottom, it is able to evaluate the type of an arbitrary identifier anywhere within a module. If the type of that identifier depends on the types of other expressions or symbols, pyright recursively evaluates those in turn until it has enough information to determine the type of the target identifier. By comparison, mypy uses a more traditional multi-pass architecture where semantic analysis is performed multiple times on a module from the top to the bottom until all types converge.
 
-Pyright implements its own parser, which recovers gracefully from syntax errors and continues parsing the remainder of the source file. By comparison, mypy uses the parser built in to the Python interpreter, and it does not support recovery after a syntax error.
+Pyright implements its own parser, which recovers gracefully from syntax errors and continues parsing the remainder of the source file. By comparison, mypy uses the parser built in to the Python interpreter, and it does not support recovery after a syntax error. This also means that when you run mypy on an older version of Python, it cannot support newer language features that require grammar changes.
 
 
 ### Type Checking Unannotated Code
@@ -29,8 +31,6 @@ By default, pyright performs type checking for all code regardless of whether it
 
 By default, mypy skips all functions or methods that do not have type annotations. This is a common source of confusion for mypy users who are surprised when type violations in unannotated functions go unreported. If the option `--check-untyped-defs` is enabled, mypy performs type checking for all functions and methods.
 
-Mypy supports the `typing.no_type_check` decorator. This decorator does not make sense for language servers, so it is ignored by pyright.
-
 
 ### Inferred Return Types
 
@@ -220,7 +220,7 @@ x = 'a'
 reveal_type(x) # pyright: Literal['a'], mypy: str
 ```
 
-Pyright also supports “literal math” for simple operations between literals.
+Pyright also supports “literal math” for simple operations involving literals.
 
 ```python
 def func1(a: Literal[1, 2], b: Literal[2, 3]):

lerna.json

@@ -2,7 +2,7 @@
     "packages": [
         "packages/*"
     ],
-    "version": "1.1.376",
+    "version": "1.1.377",
     "command": {
         "version": {
             "push": false,

packages/pyright-internal/package.json

@@ -2,7 +2,7 @@
     "name": "pyright-internal",
     "displayName": "pyright",
     "description": "Type checker for the Python language",
-    "version": "1.1.376",
+    "version": "1.1.377",
     "license": "MIT",
     "private": true,
     "files": [

packages/pyright-internal/src/analyzer/analyzerFileInfo.ts

@@ -10,7 +10,7 @@
 
 import { DiagnosticRuleSet, ExecutionEnvironment } from '../common/configOptions';
 import { TextRangeDiagnosticSink } from '../common/diagnosticSink';
-import { pythonVersion3_14 } from '../common/pythonVersion';
+import { PythonVersion, pythonVersion3_14 } from '../common/pythonVersion';
 import { TextRange } from '../common/textRange';
 import { TextRangeCollection } from '../common/textRangeCollection';
 import { Uri } from '../common/uri/uri';
@@ -79,7 +79,7 @@ export function isAnnotationEvaluationPostponed(fileInfo: AnalyzerFileInfo) {
     // release to reduce the risk. As of May 8, 2024, the change did not make it into
     // Python 3.13beta1, so it has been deferred to Python 3.14.
     // https://discuss.python.org/t/pep-649-deferred-evaluation-of-annotations-tentatively-accepted/21331
-    if (fileInfo.executionEnvironment.pythonVersion.isGreaterOrEqualTo(pythonVersion3_14)) {
+    if (PythonVersion.isGreaterOrEqualTo(fileInfo.executionEnvironment.pythonVersion, pythonVersion3_14)) {
         return true;
     }
 

packages/pyright-internal/src/analyzer/checker.ts

@@ -19,7 +19,7 @@ import { appendArray } from '../common/collectionUtils';
 import { assert, assertNever } from '../common/debug';
 import { ActionKind, Diagnostic, DiagnosticAddendum, RenameShadowedFileAction } from '../common/diagnostic';
 import { DiagnosticRule } from '../common/diagnosticRules';
-import { pythonVersion3_12, pythonVersion3_5, pythonVersion3_6 } from '../common/pythonVersion';
+import { PythonVersion, pythonVersion3_12, pythonVersion3_5, pythonVersion3_6 } from '../common/pythonVersion';
 import { TextRange } from '../common/textRange';
 import { Uri } from '../common/uri/uri';
 import { DefinitionProvider } from '../languageService/definitionProvider';
@@ -638,7 +638,7 @@ export class Checker extends ParseTreeWalker {
 
             if (
                 this._fileInfo.diagnosticRuleSet.reportTypeCommentUsage !== 'none' &&
-                this._fileInfo.executionEnvironment.pythonVersion.isGreaterOrEqualTo(pythonVersion3_5)
+                PythonVersion.isGreaterOrEqualTo(this._fileInfo.executionEnvironment.pythonVersion, pythonVersion3_5)
             ) {
                 this._evaluator.addDiagnostic(
                     DiagnosticRule.reportTypeCommentUsage,
@@ -1208,7 +1208,7 @@ export class Checker extends ParseTreeWalker {
 
             if (
                 this._fileInfo.diagnosticRuleSet.reportTypeCommentUsage !== 'none' &&
-                this._fileInfo.executionEnvironment.pythonVersion.isGreaterOrEqualTo(pythonVersion3_6)
+                PythonVersion.isGreaterOrEqualTo(this._fileInfo.executionEnvironment.pythonVersion, pythonVersion3_6)
             ) {
                 this._evaluator.addDiagnostic(
                     DiagnosticRule.reportTypeCommentUsage,
@@ -1385,7 +1385,7 @@ export class Checker extends ParseTreeWalker {
         // associated with f-strings that we need to validate. Determine whether
         // we're within an f-string (or multiple f-strings if nesting is used).
         const fStringContainers: FormatStringNode[] = [];
-        if (this._fileInfo.executionEnvironment.pythonVersion.isLessThan(pythonVersion3_12)) {
+        if (PythonVersion.isLessThan(this._fileInfo.executionEnvironment.pythonVersion, pythonVersion3_12)) {
             let curNode: ParseNode | undefined = node;
             while (curNode) {
                 if (curNode.nodeType === ParseNodeType.FormatString) {
@@ -4406,12 +4406,17 @@ export class Checker extends ParseTreeWalker {
                     (isInstantiableClass(type) && type.shared.fullName === deprecatedForm.fullName) ||
                     type.props?.typeAliasInfo?.fullName === deprecatedForm.fullName
                 ) {
-                    if (this._fileInfo.executionEnvironment.pythonVersion.isGreaterOrEqualTo(deprecatedForm.version)) {
+                    if (
+                        PythonVersion.isGreaterOrEqualTo(
+                            this._fileInfo.executionEnvironment.pythonVersion,
+                            deprecatedForm.version
+                        )
+                    ) {
                         if (!deprecatedForm.typingImportOnly || isImportFromTyping) {
                             this._evaluator.addDiagnostic(
                                 DiagnosticRule.reportDeprecated,
                                 LocMessage.deprecatedType().format({
-                                    version: deprecatedForm.version.toString(),
+                                    version: PythonVersion.toString(deprecatedForm.version),
                                     replacement: deprecatedForm.replacementText,
                                 }),
                                 node

packages/pyright-internal/src/analyzer/codeFlowEngine.ts

@@ -1299,7 +1299,6 @@ export function getCodeFlowEngine(
                     (FlowFlags.VariableAnnotation |
                         FlowFlags.Assignment |
                         FlowFlags.WildcardImport |
-                        FlowFlags.NarrowForPattern |
                         FlowFlags.ExhaustedMatch)
                 ) {
                     const typedFlowNode = curFlowNode as
@@ -1311,6 +1310,25 @@ export function getCodeFlowEngine(
                     continue;
                 }
 
+                if (curFlowNode.flags & FlowFlags.NarrowForPattern) {
+                    const patternFlowNode = curFlowNode as FlowNarrowForPattern;
+
+                    const typeResult = evaluator.evaluateTypeForSubnode(patternFlowNode.statement, () => {
+                        if (patternFlowNode.statement.nodeType === ParseNodeType.Case) {
+                            evaluator.evaluateTypesForCaseStatement(patternFlowNode.statement);
+                        } else {
+                            evaluator.evaluateTypesForMatchStatement(patternFlowNode.statement);
+                        }
+                    });
+
+                    if (typeResult && isNever(typeResult.type)) {
+                        return cacheReachabilityResult(Reachability.UnreachableByAnalysis);
+                    }
+
+                    curFlowNode = patternFlowNode.antecedent;
+                    continue;
+                }
+
                 if (
                     curFlowNode.flags &
                     (FlowFlags.TrueCondition |

packages/pyright-internal/src/analyzer/dataClasses.ts

@@ -11,7 +11,7 @@
 import { assert } from '../common/debug';
 import { DiagnosticAddendum } from '../common/diagnostic';
 import { DiagnosticRule } from '../common/diagnosticRules';
-import { pythonVersion3_13 } from '../common/pythonVersion';
+import { PythonVersion, pythonVersion3_13 } from '../common/pythonVersion';
 import { LocMessage } from '../localization/localize';
 import {
     ArgCategory,
@@ -122,7 +122,12 @@ export function synthesizeDataClassMethods(
 
     // For Python 3.13 and newer, synthesize a __replace__ method.
     let replaceType: FunctionType | undefined;
-    if (AnalyzerNodeInfo.getFileInfo(node).executionEnvironment.pythonVersion >= pythonVersion3_13) {
+    if (
+        PythonVersion.isGreaterOrEqualTo(
+            AnalyzerNodeInfo.getFileInfo(node).executionEnvironment.pythonVersion,
+            pythonVersion3_13
+        )
+    ) {
         replaceType = FunctionType.createSynthesizedInstance('__replace__');
         FunctionType.addParam(replaceType, selfParam);
         FunctionType.addKeywordOnlyParamSeparator(replaceType);
@@ -208,6 +213,7 @@ export function synthesizeDataClassMethods(
             let variableTypeEvaluator: EntryTypeEvaluator | undefined;
             let hasDefaultValue = false;
             let isKeywordOnly = ClassType.isDataClassKeywordOnly(classType) || sawKeywordOnlySeparator;
+            let defaultExpr: ExpressionNode | undefined;
             let includeInInit = true;
             let converter: ArgumentNode | undefined;
 
@@ -230,6 +236,7 @@ export function synthesizeDataClassMethods(
                 }
 
                 hasDefaultValue = true;
+                defaultExpr = statement.d.rightExpr;
 
                 // If the RHS of the assignment is assigning a field instance where the
                 // "init" parameter is set to false, do not include it in the init method.
@@ -293,6 +300,9 @@ export function synthesizeDataClassMethods(
                         );
 
                         hasDefaultValue = !!defaultArg;
+                        if (defaultArg?.d.valueExpr) {
+                            defaultExpr = defaultArg.d.valueExpr;
+                        }
 
                         const aliasArg = statement.d.rightExpr.d.args.find((arg) => arg.d.name?.d.value === 'alias');
                         if (aliasArg) {
@@ -360,6 +370,7 @@ export function synthesizeDataClassMethods(
                         alias: aliasName,
                         isKeywordOnly: false,
                         hasDefault: hasDefaultValue,
+                        defaultExpr,
                         includeInInit,
                         nameNode: variableNameNode,
                         type: UnknownType.create(),
@@ -377,6 +388,7 @@ export function synthesizeDataClassMethods(
                         alias: aliasName,
                         isKeywordOnly,
                         hasDefault: hasDefaultValue,
+                        defaultExpr,
                         includeInInit,
                         nameNode: variableNameNode,
                         type: UnknownType.create(),
@@ -402,6 +414,7 @@ export function synthesizeDataClassMethods(
                         // causes overridden variables to "inherit" default values from parent classes.
                         if (!dataClassEntry.hasDefault && oldEntry.hasDefault && oldEntry.includeInInit) {
                             dataClassEntry.hasDefault = true;
+                            dataClassEntry.defaultExpr = oldEntry.defaultExpr;
                             hasDefaultValue = true;
 
                             // Warn the user of this case because it can result in type errors if the
@@ -541,7 +554,8 @@ export function synthesizeDataClassMethods(
                         effectiveType,
                         FunctionParamFlags.TypeDeclared,
                         effectiveName,
-                        entry.hasDefault ? entry.type : undefined
+                        entry.hasDefault ? entry.type : undefined,
+                        entry.defaultExpr
                     );
 
                     if (entry.isKeywordOnly) {

packages/pyright-internal/src/analyzer/enums.ts

@@ -8,7 +8,7 @@
  */
 
 import { assert } from '../common/debug';
-import { pythonVersion3_13 } from '../common/pythonVersion';
+import { PythonVersion, pythonVersion3_13 } from '../common/pythonVersion';
 import { ArgCategory, ExpressionNode, NameNode, ParseNode, ParseNodeType } from '../parser/parseNodes';
 import { getFileInfo } from './analyzerNodeInfo';
 import { VariableDeclaration } from './declaration';
@@ -413,7 +413,10 @@ export function transformTypeForEnumMember(
             // are treated as members.
             if (isInstantiableClass(assignedType)) {
                 const fileInfo = getFileInfo(primaryDecl.node);
-                isMemberOfEnumeration = fileInfo.executionEnvironment.pythonVersion.isLessThan(pythonVersion3_13);
+                isMemberOfEnumeration = PythonVersion.isLessThan(
+                    fileInfo.executionEnvironment.pythonVersion,
+                    pythonVersion3_13
+                );
             }
         }
     }

packages/pyright-internal/src/analyzer/importResolver.ts

@@ -472,7 +472,7 @@ export class ImportResolver {
         this._cachedTypeshedStdLibModuleVersionInfo.forEach((versionInfo, moduleName) => {
             let shouldExcludeModule = false;
 
-            if (versionInfo.max !== undefined && pythonVersion.isGreaterThan(versionInfo.max)) {
+            if (versionInfo.max !== undefined && PythonVersion.isGreaterThan(pythonVersion, versionInfo.max)) {
                 shouldExcludeModule = true;
             }
 
@@ -2008,11 +2008,11 @@ export class ImportResolver {
             const versionInfo = this._cachedTypeshedStdLibModuleVersionInfo.get(namePartsToConsider.join('.'));
 
             if (versionInfo) {
-                if (pythonVersion.isLessThan(versionInfo.min)) {
+                if (PythonVersion.isLessThan(pythonVersion, versionInfo.min)) {
                     return false;
                 }
 
-                if (versionInfo.max !== undefined && pythonVersion.isGreaterThan(versionInfo.max)) {
+                if (versionInfo.max !== undefined && PythonVersion.isGreaterThan(pythonVersion, versionInfo.max)) {
                     return false;
                 }