logistic regression

by Emanuele Battaglia

The project discusses the conceptual and practical aspects of credit scoring using logistic regression, gradually bridging to neural networks. The focus is on clarity of explanation and the fact that logistic regression is just a simple neural network.

deliverables

1. Introduction to Credit Scoring and the Logistic Function

Summarizing foundational ideas and interpret key formulas.

Requirements

• Read and summarize the Credit Scoring section from the course notes.
• Define the logistic (sigmoid) function, including its
  • graphical shape and interpretation, and
  • why it is suitable for modeling probabilities.
• Summarize the discussion between Alice and Bob. What disagreement or problem is posed? How is it resolved mathematically?
• Explain the \(\log-\)likelihood function:
  • What does it measure?
  • Why do we minimize the negative \(\log-\)likelihood?
  • What does this have to do with training a model?

2. Logistic Regression Task

Apply logistic regression to a binary credit scoring dataset and interpret the model’s behavior.

Requirements

• Load the dataset from Kaggle: Credit Score Classification Dataset.¹
• Use LogisticRegression from sklearn.linear_model to fit the model.
• Visualize the sigmoid function using numpy and matplotlib.
• Plot the decision boundary using contourf and interpret its meaning.
• Use predict_proba, score, and confusion_matrix to evaluate performance.
• Finally, (try to) include explanatory markdown cells to describe each step and its rationale.

3. From Logistic Regression to Neural Networks

Requirements

• Is logistic regression equivalent to a neural network with no hidden layers and sigmoid activation?
• Describe how adding hidden layers and neurons increases model capacity.
• Discuss:
  • Why use neural networks if logistic regression already works?
  • What limitations does logistic regression have?

Resources

• Credit Scoring. Course Notes.
• Understanding Logistic Regression. GeeksforGeeks. (Advanced)
• Logistic Regression is a Neural Network. TDS.
• What is a Neural Network?. 3Blue1Brown.